Notes from November 17th, 2023

Weekly meeting of the TDP contributors to discuss opens PR and particularly the ones related to the TDP 1.1 release.

Pull Requests

Weekly review of open PRs (in chronological order):

  • TDP#90: For now, build have been tested for :

    • Hadoop 3.1.4
    • Tez 0.9.1
    • Hive 1.2.3
    • Spark 2.3.5
    • Hive 2.3.9

    Some issues remain for:

    • Spark 3.2.4 see spark#3
    • HBase Connectors, which can’t be build because it needs Spark 3 jars
    • HBase 2.1.10 is not compatible with Hadoop 3.1.4 because of a dependency upgrade (jetty) that is not backported on 2.1.10 (EOL and shaded in later versions). @Pierrotws is working on a PR to upgrade jetty on HBase 2.1.10.
  • spark#3: Issue with mvn install (see spark-old#3). We need to try to buid Spark 3.2.4 with and without this PR to see if it improve the test output. Spark 2 is ok.

  • ranger#4: HBase plugin performance improvements that has not been released by Ranger but could be used. Merged.

  • phoenix#5: Merged. Build can be tested now.

  • phoenix-queryserver#5 Merged. Build can be tested now.

  • knox#3: Merged. Build can be tested now.

  • hbase-connectors#3: Merged. Build can be tested now.

  • hbase-operator-tools#3: Merged. Build can be tested now.

Open topics

  • @sergkudinov, related to this discussion: PR can be opened to reflect the suggested updates. tdp-cheatsheet repository visibility has be set to private.
  • @PaulFarault, about the future of the tdp-getting-started repository: tdp-getting-started will be archived along with tdp-vagrant. Content of tdp-getting-started dev branch and tdp-vagrant will be imported to tdp-dev. tdp-dev will still contain Git submodules and the script will be splitted in several utility scripts.
  • @PaulFarault, about repository names: Discussion about how to name the collections repository should be updated in order to take a decision.
  • @PACordonnier: JMX exported metrics names are not user friendly. It would be nice to have a way to rewrite them using a pattern. See this article. A PR should be opened to discuss this topic.