java - Spring Cloud Dataflow本地使用dockerized任务

Question

我正在使用 Pivotal 在其网站上提供的 de docker-compose，当尝试使用 Docker 版本的时间戳启动任务时，日志显示以下消息:

dataflow-server    | 2020-06-15 07:48:51.867  INFO 1 --- [io-9393-exec-10] o.s.c.d.spi.local.LocalTaskLauncher      : Preparing to run an application from Docker Resource [docker:springcloudtask/timestamp-task:2.1.1.RELEASE]. This may take some time if the artifact must be downloaded from a remote host.
dataflow-server    | 2020-06-15 07:48:51.872  INFO 1 --- [io-9393-exec-10] o.s.c.d.spi.local.LocalTaskLauncher      : Command to be executed: docker run -e SPRING_APPLICATION_JSON={"endpoints.jmx.unique-names":"true","endpoints.shutdown.enabled":"true","spring.datasource.username":"postgres","spring.datasource.url":"jdbc:postgresql://postgres:5432/dataflow","spring.datasource.driverClassName":"org.postgresql.Driver","server.port":"41226","spring.cloud.task.name":"TareaDocker","spring.datasource.password":"postgres","spring.jmx.default-domain":"TareaDocker-55685c2e-ad6a-4938-8f83-6708877d728c"} -p 41226:41226 springcloudtask/timestamp-task:2.1.1.RELEASE --spring.cloud.data.flow.platformname=default --spring.cloud.task.executionid=1
dataflow-server    | 2020-06-15 07:48:51.881 ERROR 1 --- [io-9393-exec-10] o.s.c.d.s.c.RestControllerAdvice         : Caught exception while handling a request
dataflow-server    |
dataflow-server    | java.lang.RuntimeException: Exception trying to launch [AppDeploymentRequest@5fb36004 commandlineArguments = list['--spring.cloud.data.flow.platformname=default', '--spring.cloud.task.executionid=1'], deploymentProperties = map[[empty]], definition = [AppDefinition@27fe9390 name = 'TareaDocker', properties = map['spring.datasource.username' -> 'postgres', 'spring.cloud.task.name' -> 'TareaDocker', 'spring.datasource.url' -> 'jdbc:postgresql://postgres:5432/dataflow', 'spring.datasource.driverClassName' -> 'org.postgresql.Driver', 'spring.datasource.password' -> 'postgres']], resource = Docker Resource [docker:springcloudtask/timestamp-task:2.1.1.RELEASE]]
dataflow-server    |    at org.springframework.cloud.deployer.spi.local.LocalTaskLauncher.launch(LocalTaskLauncher.java:126)
dataflow-server    |    at org.springframework.cloud.dataflow.server.service.impl.DefaultTaskExecutionService.executeTask(DefaultTaskExecutionService.java:293)
dataflow-server    |    at org.springframework.cloud.dataflow.server.service.impl.DefaultTaskExecutionService$$FastClassBySpringCGLIB$$422cda43.invoke(<generated>)
dataflow-server    |    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
dataflow-server    |    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
dataflow-server    |    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
dataflow-server    |    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
dataflow-server    |    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366)
dataflow-server    |    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
dataflow-server    |    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
dataflow-server    |    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
dataflow-server    |    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
dataflow-server    |    at org.springframework.cloud.dataflow.server.service.impl.DefaultTaskExecutionService$$EnhancerBySpringCGLIB$$4c40935a.executeTask(<generated>)
dataflow-server    |    at org.springframework.cloud.dataflow.server.controller.TaskExecutionController.launch(TaskExecutionController.java:172)
dataflow-server    |    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
dataflow-server    |    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
dataflow-server    |    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
dataflow-server    |    at java.lang.reflect.Method.invoke(Method.java:498)
dataflow-server    |    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
dataflow-server    |    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
dataflow-server    |    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
dataflow-server    |    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
dataflow-server    |    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
dataflow-server    |    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
dataflow-server    |    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
dataflow-server    |    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
dataflow-server    |    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
dataflow-server    |    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
dataflow-server    |    at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
dataflow-server    |    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
dataflow-server    |    at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
dataflow-server    |    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
dataflow-server    |    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
dataflow-server    |    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
dataflow-server    |    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
dataflow-server    |    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
dataflow-server    |    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:103)
dataflow-server    |    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
dataflow-server    |    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
dataflow-server    |    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
dataflow-server    |    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
dataflow-server    |    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
dataflow-server    |    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
dataflow-server    |    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
dataflow-server    |    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
dataflow-server    |    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
dataflow-server    |    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
dataflow-server    |    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:109)
dataflow-server    |    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
dataflow-server    |    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
dataflow-server    |    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
dataflow-server    |    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
dataflow-server    |    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
dataflow-server    |    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
dataflow-server    |    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
dataflow-server    |    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
dataflow-server    |    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
dataflow-server    |    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
dataflow-server    |    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
dataflow-server    |    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
dataflow-server    |    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
dataflow-server    |    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
dataflow-server    |    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373)
dataflow-server    |    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
dataflow-server    |    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
dataflow-server    |    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590)
dataflow-server    |    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
dataflow-server    |    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
dataflow-server    |    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
dataflow-server    |    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
dataflow-server    |    at java.lang.Thread.run(Thread.java:748)
dataflow-server    | Caused by: java.io.IOException: Cannot run program "docker" (in directory "/tmp/88265136092561/TareaDocker-55685c2e-ad6a-4938-8f83-6708877d728c"): error=2, No such file or directory
dataflow-server    |    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
dataflow-server    |    at org.springframework.cloud.deployer.spi.local.LocalTaskLauncher$TaskInstance.start(LocalTaskLauncher.java:340)
dataflow-server    |    at org.springframework.cloud.deployer.spi.local.LocalTaskLauncher$TaskInstance.access$200(LocalTaskLauncher.java:244)
dataflow-server    |    at org.springframework.cloud.deployer.spi.local.LocalTaskLauncher.launch(LocalTaskLauncher.java:119)
dataflow-server    |    ... 70 common frames omitted
dataflow-server    | Caused by: java.io.IOException: error=2, No such file or directory
dataflow-server    |    at java.lang.UNIXProcess.forkAndExec(Native Method)
dataflow-server    |    at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
dataflow-server    |    at java.lang.ProcessImpl.start(ProcessImpl.java:134)
dataflow-server    |    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
dataflow-server    |    ... 73 common frames omitted
dataflow-server    |

我已访问 CLI，但找不到 docker。我将 Docker 安装到镜像中，重新启动时日志文件显示:

docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.

我无法启动 Docker 守护进程，因为当我尝试以 root 身份时，系统说我必须是 root:

root@5e1fc09ff833:/var/log# systemctl enable dockerd
bash: systemctl: command not found

root@5e1fc09ff833:/tmp/104228715709066/TaskDocker-c7e7a8ec-5d1e-4b23-97d2-5f56114102fe# dockerd

....
**failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to create NAT chain DOCKER: iptables failed: iptables -t nat -N DOCKER: iptables v1.6.1: can't initialize iptables table `nat': Permission denied (you must be root)
Perhaps iptables or your kernel needs to be upgraded.
 (exit status 3)**

是否可以在本地模式下使用 Spring Cloud Dataflow 启动 Docker 任务？

谢谢

Answer 1

正如我们在 Gitter 中已经讨论过的那样，目前在本地部署上暂时支持 Docker 镜像，并且在 SCDF 2.6.x 发行版中提供。我们最近也发布了 2.6 M1。

但是，更强大的解决方案正在进行中，您可以在 spring-cloud/spring-cloud-deployer-local#185 上跟踪进度。 .

附注:展望 future ，如果您不再在多个论坛中重复发布相同的问题，那就太好了。