Publish to my blog (weekly)

    • If you run your driver inside a Kubernetes pod, you can use a headless service to allow your driver pod to be routable from the executors by a stable hostname.
    • When deploying your headless service, ensure that the service’s label selector will only match the driver pod and no other pods; it is recommended to assign your driver pod a sufficiently unique label and to use that label in the label selector of the headless service.
    • Specify the driver’s hostname via spark.driver.host and your spark driver’s port to spark.driver.port.

Posted from Diigo. The rest of my favorite links are here.

댓글

이 블로그의 인기 게시물

Publish to my blog (weekly)

Publish to my blog (weekly)