spark rename or access columns which has special chars " ?:

I have a columns like below root |-- metadata: struct (nullable = true) ||-- "drop":{"dropPath":" https://dstpath.media27.ec2.st-av.net/drop?source_id: string (nullable = true) ||-- "selection":{"AlllURL":" https://dstpath.media27.ec2.st-av.net/image?source_id: string (

Re: Live Streamed Code Review today at 11am Pacific

This afternoon @ 3pm pacific I'll be looking at review tooling for Spark & Beam https://www.youtube.com/watch?v=ff8_jbzC8JI. Next week's regular Friday code (this time July 20th @ 9:30am pacific) review will once again probably have more of an ML focus for folks interested in watching Spark ML PRs

[build system] spark/k8s integration tests are now working

after upgrading minikube to v0.28.0 and much wailing and gnashing of teeth, it was discovered that v0.25.0 actually *works* as expected and the k8s integration tests are now green! side note, i've also opportunistically upgraded the minikube VM drivers from kvm to kvm2. shane -- Shane Knapp UC B