Hi folks,
Just a note to say that the latest Bigtop charms are now available on the Edge channel.
This include updated:
Hadoop Namenode
I have an updated Spark charm I’m testing locally and will update on the edge channel shortly which will update Spark to 2.2.3
This bumps our Bigtop release from 1.2.x to 1.4.0 and the release notes can be found here:https://blogs.apache.org/bigtop/entry/release-of-apache-bigtop-13
Going forward I will update the charms for Juju Storage and Network support and will look to bring in the K8S Hadoop charms in line with the Bigtop community activity, I dont’ see much value in rolling our own I’d rather have a few other developers helping out so I shall lend a hand upstream and bring them down to Juju when ready.
Anyway… I’ll leave them in edge and update the bundles and test a few things over the next week or so and slowly bump them through the channels. Unless someone says otherwise I’m gonna run a lazy consensus on this one and if no one squeals I’ll push them into stable in a week or so.
Tom
3 Likes
Sorry for the delay, Spark on Edge channel is 2.2.3!
1 Like
Hive is now at 2.3.3
both on the edge channel. I’ve not tested these yet so YMMV.
1 Like
kwmonroe
4 November 2019 11:54
4
Still running the lazy consensus? I paid down some technical debt related to bigtop:
BIGTOP-3015 – remove client from hadoop-spark
And the grand finale:
BIGTOP-3271 (update all versions/revs; @magicaltrout , i could use a +1 on this )
The bigtop-repo resource has been updated to branch-1.4 as bigtop-repo-8 and relevant artifacts are in edge:
url: cs:~bigdata-charmers/hadoop-namenode-54
url: cs:~bigdata-charmers/hadoop-plugin-52
url: cs:~bigdata-charmers/hadoop-resourcemanager-56
url: cs:~bigdata-charmers/hadoop-slave-55
url: cs:~bigdata-charmers/hbase-45
url: cs:~bigdata-charmers/hive-35
url: cs:~bigdata-charmers/kafka-55
url: cs:~bigdata-charmers/mahout-43
url: cs:~bigdata-charmers/spark-81
url: cs:~bigdata-charmers/zeppelin-52
url: cs:~bigdata-charmers/zookeeper-58
url: cs:~bigdata-charmers/bundle/hadoop-hbase-48
url: cs:~bigdata-charmers/bundle/hadoop-kafka-63
url: cs:~bigdata-charmers/bundle/hadoop-processing-92
url: cs:~bigdata-charmers/bundle/hadoop-spark-66
url: cs:~bigdata-charmers/bundle/spark-processing-84
Smoke tests across all bundles looks good, eg:
$ juju run-action --wait hbase/0 smoke-test
unit-hbase-0:
id: 180a247f-0c3a-418d-86bc-6150ebd963eb
results:
outcome: success
status: completed
timing:
completed: 2019-11-02 14:35:56 +0000 UTC
enqueued: 2019-11-02 14:35:30 +0000 UTC
started: 2019-11-02 14:35:34 +0000 UTC
unit: hbase/0
$ juju run-action --wait namenode/0 smoke-test
unit-namenode-0:
id: ac6b89a7-e119-4cbe-8e30-f6acadc1be81
results:
outcome: success
status: completed
timing:
completed: 2019-11-02 14:36:18 +0000 UTC
enqueued: 2019-11-02 14:36:03 +0000 UTC
started: 2019-11-02 14:36:03 +0000 UTC
unit: namenode/0
$ juju run-action --wait resourcemanager/0 smoke-test
unit-resourcemanager-0:
id: 2cd09d1c-2fd5-4498-831e-980f511c3c29
results:
outcome: success
status: completed
timing:
completed: 2019-11-02 14:37:22 +0000 UTC
enqueued: 2019-11-02 14:36:28 +0000 UTC
started: 2019-11-02 14:36:32 +0000 UTC
unit: resourcemanager/0
$ juju run-action --wait zookeeper/0 smoke-test
unit-zookeeper-0:
id: 8d7070a3-e00a-447f-896b-b5d7d1cc72e2
results:
outcome: success
status: completed
timing:
completed: 2019-11-02 14:38:06 +0000 UTC
enqueued: 2019-11-02 14:37:32 +0000 UTC
started: 2019-11-02 14:37:36 +0000 UTC
unit: zookeeper/0
I also added my release scripts to the bigtop base layer in case they’re helpful. Lmk if you want me to release through additional channels.
2 Likes
You gazumped my release!
2 Likes
