- html - 出于某种原因,IE8 对我的 Sass 文件中继承的 html5 CSS 不友好?
- JMeter 在响应断言中使用 span 标签的问题
- html - 在 :hover and :active? 上具有不同效果的 CSS 动画
- html - 相对于居中的 html 内容固定的 CSS 重复背景?
我对 Hadoop 和 Docker 很陌生,我正在按照本教程使用 docker 容器运行 Hadoop 集群。
来源:https://clubhouse.io/developer-how-to/how-to-set-up-a-hadoop-cluster-in-docker/
当我键入以下内容时,就像在教程中一样:
docker ps
我得到以下输出:
docker-compose.yml
的内容:
version: "2"
services:
namenode:
build: ./namenode
image: bde2020/hadoop-namenode:1.1.0-hadoop2.7.1-java8
container_name: namenode
volumes:
- hadoop_namenode:/hadoop/dfs/name
environment:
- CLUSTER_NAME=test
env_file:
- ./hadoop.env
ports:
- "9870:50070"
resourcemanager:
build: ./resourcemanager
image: bde2020/hadoop-resourcemanager:1.1.0-hadoop2.7.1-java8
container_name: resourcemanager
depends_on:
- namenode
- datanode1
- datanode2
env_file:
- ./hadoop.env
ports:
- "8089:8088"
historyserver:
build: ./historyserver
image: bde2020/hadoop-historyserver:1.1.0-hadoop2.7.1-java8
container_name: historyserver
depends_on:
- namenode
- datanode1
- datanode2
volumes:
- hadoop_historyserver:/hadoop/yarn/timeline
env_file:
- ./hadoop.env
nodemanager1:
build: ./nodemanager
image: bde2020/hadoop-nodemanager:1.1.0-hadoop2.7.1-java8
container_name: nodemanager1
depends_on:
- namenode
- datanode1
- datanode2
env_file:
- ./hadoop.env
datanode1:
build: ./datanode
image: bde2020/hadoop-datanode:1.1.0-hadoop2.7.1-java8
container_name: datanode1
depends_on:
- namenode
volumes:
- hadoop_datanode1:/hadoop/dfs/data
env_file:
- ./hadoop.env
datanode2:
build: ./datanode
image: bde2020/hadoop-datanode:1.1.0-hadoop2.7.1-java8
container_name: datanode2
depends_on:
- namenode
volumes:
- hadoop_datanode2:/hadoop/dfs/data
env_file:
- ./hadoop.env
datanode3:
build: ./datanode
image: bde2020/hadoop-datanode:1.1.0-hadoop2.7.1-java8
container_name: datanode3
depends_on:
- namenode
volumes:
- hadoop_datanode3:/hadoop/dfs/data
env_file:
- ./hadoop.env
volumes:
hadoop_namenode:
hadoop_datanode1:
hadoop_datanode2:
hadoop_datanode3:
hadoop_historyserver:
实际上,即使在运行之后:
docker run -d -p 80:80 --name myserver nginx
本地主机浏览器中没有显示
$ docker logs f8e90944fedf
Configuring core
- Setting hadoop.proxyuser.hue.hosts=*
- Setting fs.defaultFS=hdfs://namenode:9000
- Setting hadoop.http.staticuser.user=root
- Setting io.compression.codecs=org.apache.hadoop.io.compress.SnappyCodec
- Setting hadoop.proxyuser.hue.groups=*
Configuring hdfs
- Setting dfs.namenode.datanode.registration.ip-hostname-check=false
- Setting dfs.webhdfs.enabled=true
- Setting dfs.permissions.enabled=false
- Setting dfs.namenode.name.dir=file:///hadoop/dfs/name
Configuring yarn
- Setting yarn.timeline-service.enabled=true
- Setting yarn.scheduler.capacity.root.default.maximum-allocation-vcores=4
- Setting yarn.resourcemanager.system-metrics-publisher.enabled=true
- Setting yarn.resourcemanager.store.class=org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
- Setting yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage=98.5
- Setting yarn.log.server.url=http://historyserver:8188/applicationhistory/logs/
- Setting yarn.resourcemanager.fs.state-store.uri=/rmstate
- Setting yarn.timeline-service.generic-application-history.enabled=true
- Setting yarn.log-aggregation-enable=true
- Setting yarn.resourcemanager.hostname=resourcemanager
- Setting yarn.scheduler.capacity.root.default.maximum-allocation-mb=8192
- Setting yarn.nodemanager.aux-services=mapreduce_shuffle
- Setting yarn.resourcemanager.resource_tracker.address=resourcemanager:8031
- Setting yarn.timeline-service.hostname=historyserver
- Setting yarn.resourcemanager.scheduler.address=resourcemanager:8030
- Setting yarn.resourcemanager.address=resourcemanager:8032
- Setting mapred.map.output.compress.codec=org.apache.hadoop.io.compress.SnappyCodec
- Setting yarn.nodemanager.remote-app-log-dir=/app-logs
- Setting yarn.resourcemanager.scheduler.class=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
- Setting mapreduce.map.output.compress=true
- Setting yarn.nodemanager.resource.memory-mb=16384
- Setting yarn.resourcemanager.recovery.enabled=true
- Setting yarn.nodemanager.resource.cpu-vcores=8
Configuring httpfs
Configuring kms
Configuring mapred
- Setting mapreduce.map.java.opts=-Xmx3072m
- Setting mapreduce.reduce.java.opts=-Xmx6144m
- Setting mapreduce.reduce.memory.mb=8192
- Setting yarn.app.mapreduce.am.env=HADOOP_MAPRED_HOME=/opt/hadoop-3.2.1/
- Setting mapreduce.map.memory.mb=4096
- Setting mapred.child.java.opts=-Xmx4096m
- Setting mapreduce.reduce.env=HADOOP_MAPRED_HOME=/opt/hadoop-3.2.1/
- Setting mapreduce.framework.name=yarn
- Setting mapreduce.map.env=HADOOP_MAPRED_HOME=/opt/hadoop-3.2.1/
Configuring for multihomed network
remove lost+found from /hadoop/dfs/name
rm: cannot remove '/hadoop/dfs/name/lost+found': No such file or directory
2020-08-10 10:53:49,592 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = f8e90944fedf/172.22.0.2
STARTUP_MSG: args = []
STARTUP_MSG: version = 3.2.1
这是一个关于类路径的大代码块(我删除了它,否则无法发布)。这是其余的:
STARTUP_MSG: build = https://gitbox.apache.org/repos/asf/hadoop.git -r b3cbbb467e22ea829b3808f4b7b01d07e0bf3842; compiled by 'rohithsharmaks' on 2019-09-10T15:56Z
STARTUP_MSG: java = 1.8.0_232
************************************************************/
2020-08-10 10:53:50,063 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
2020-08-10 10:53:55,494 INFO namenode.NameNode: createNameNode []
2020-08-10 10:54:01,381 INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties
2020-08-10 10:54:05,032 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
2020-08-10 10:54:05,056 INFO impl.MetricsSystemImpl: NameNode metrics system started
2020-08-10 10:54:05,617 INFO namenode.NameNodeUtils: fs.defaultFS is hdfs://namenode:9000
2020-08-10 10:54:05,623 INFO namenode.NameNode: Clients should use namenode:9000 to access this namenode/service.
2020-08-10 10:54:10,545 INFO util.JvmPauseMonitor: Starting JVM pause monitor
2020-08-10 10:54:11,572 INFO hdfs.DFSUtil: Starting Web-server for hdfs at: http://0.0.0.0:9870
2020-08-10 10:54:11,964 INFO util.log: Logging initialized @37617ms
2020-08-10 10:54:15,166 INFO server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
2020-08-10 10:54:15,510 INFO http.HttpRequestLog: Http request log for http.requests.namenode is not defined
2020-08-10 10:54:15,814 INFO http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2020-08-10 10:54:15,898 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
2020-08-10 10:54:15,899 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
2020-08-10 10:54:15,903 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
2020-08-10 10:54:16,378 INFO http.HttpServer2: Added filter 'org.apache.hadoop.hdfs.web.AuthFilter' (class=org.apache.hadoop.hdfs.web.AuthFilter)
2020-08-10 10:54:16,386 INFO http.HttpServer2: addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
2020-08-10 10:54:16,704 INFO http.HttpServer2: Jetty bound to port 9870
2020-08-10 10:54:16,720 INFO server.Server: jetty-9.3.24.v20180605, build timestamp: 2018-06-05T17:11:56Z, git hash: 84205aa28f11a4f31f2a3b86d1bba2cc8ab69827
2020-08-10 10:54:17,880 INFO handler.ContextHandler: Started o.e.j.s.ServletContextHandler@7d0b7e3c{/logs,file:///opt/hadoop-3.2.1/logs/,AVAILABLE}
2020-08-10 10:54:17,910 INFO handler.ContextHandler: Started o.e.j.s.ServletContextHandler@4b741d6d{/static,file:///opt/hadoop-3.2.1/share/hadoop/hdfs/webapps/static/,AVAILABLE}
2020-08-10 10:54:21,077 INFO handler.ContextHandler: Started o.e.j.w.WebAppContext@7486b455{/,file:///opt/hadoop-3.2.1/share/hadoop/hdfs/webapps/hdfs/,AVAILABLE}{/hdfs}
2020-08-10 10:54:21,250 INFO server.AbstractConnector: Started ServerConnector@6de0a3f9{HTTP/1.1,[http/1.1]}{0.0.0.0:9870}
2020-08-10 10:54:21,268 INFO server.Server: Started @46920ms
2020-08-10 10:54:29,608 WARN namenode.FSNamesystem: Only one image storage directory (dfs.namenode.name.dir) configured. Beware of data loss due to lack of redundant storage directories!
2020-08-10 10:54:29,609 WARN namenode.FSNamesystem: Only one namespace edits storage directory (dfs.namenode.edits.dir) configured. Beware of data loss due to lack of redundant storage directories!
2020-08-10 10:54:34,440 INFO namenode.FSEditLog: Edit logging is async:true
2020-08-10 10:54:35,312 INFO namenode.FSNamesystem: KeyProvider: null
2020-08-10 10:54:35,490 INFO namenode.FSNamesystem: fsLock is fair: true
2020-08-10 10:54:35,570 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false
2020-08-10 10:54:36,027 INFO namenode.FSNamesystem: fsOwner = root (auth:SIMPLE)
2020-08-10 10:54:36,034 INFO namenode.FSNamesystem: supergroup = supergroup
2020-08-10 10:54:36,035 INFO namenode.FSNamesystem: isPermissionEnabled = false
2020-08-10 10:54:36,035 INFO namenode.FSNamesystem: HA Enabled: false
2020-08-10 10:54:37,237 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
2020-08-10 10:54:37,572 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000
2020-08-10 10:54:37,576 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=false
2020-08-10 10:54:37,758 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
2020-08-10 10:54:37,774 INFO blockmanagement.BlockManager: The block deletion will start around 2020 Aug 10 10:54:37
2020-08-10 10:54:37,816 INFO util.GSet: Computing capacity for map BlocksMap
2020-08-10 10:54:37,822 INFO util.GSet: VM type = 64-bit
2020-08-10 10:54:37,849 INFO util.GSet: 2.0% max memory 239.8 MB = 4.8 MB
2020-08-10 10:54:37,854 INFO util.GSet: capacity = 2^19 = 524288 entries
2020-08-10 10:54:38,122 INFO blockmanagement.BlockManager: Storage policy satisfier is disabled
2020-08-10 10:54:38,126 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false
2020-08-10 10:54:38,293 INFO Configuration.deprecation: No unit for dfs.namenode.safemode.extension(30000) assuming MILLISECONDS
2020-08-10 10:54:38,295 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
2020-08-10 10:54:38,298 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0
2020-08-10 10:54:38,299 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000
2020-08-10 10:54:38,301 INFO blockmanagement.BlockManager: defaultReplication = 3
2020-08-10 10:54:38,311 INFO blockmanagement.BlockManager: maxReplication = 512
2020-08-10 10:54:38,312 INFO blockmanagement.BlockManager: minReplication = 1
2020-08-10 10:54:38,313 INFO blockmanagement.BlockManager: maxReplicationStreams = 2
2020-08-10 10:54:38,313 INFO blockmanagement.BlockManager: redundancyRecheckInterval = 3000ms
2020-08-10 10:54:38,317 INFO blockmanagement.BlockManager: encryptDataTransfer = false
2020-08-10 10:54:38,317 INFO blockmanagement.BlockManager: maxNumBlocksToLog = 1000
2020-08-10 10:54:39,701 INFO namenode.FSDirectory: GLOBAL serial map: bits=29 maxEntries=536870911
2020-08-10 10:54:39,705 INFO namenode.FSDirectory: USER serial map: bits=24 maxEntries=16777215
2020-08-10 10:54:39,706 INFO namenode.FSDirectory: GROUP serial map: bits=24 maxEntries=16777215
2020-08-10 10:54:39,707 INFO namenode.FSDirectory: XATTR serial map: bits=24 maxEntries=16777215
2020-08-10 10:54:40,102 INFO util.GSet: Computing capacity for map INodeMap
2020-08-10 10:54:40,116 INFO util.GSet: VM type = 64-bit
2020-08-10 10:54:40,172 INFO util.GSet: 1.0% max memory 239.8 MB = 2.4 MB
2020-08-10 10:54:40,210 INFO util.GSet: capacity = 2^18 = 262144 entries
2020-08-10 10:54:40,322 INFO namenode.FSDirectory: ACLs enabled? false
2020-08-10 10:54:40,328 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true
2020-08-10 10:54:40,330 INFO namenode.FSDirectory: XAttrs enabled? true
2020-08-10 10:54:40,331 INFO namenode.NameNode: Caching file names occurring more than 10 times
2020-08-10 10:54:40,638 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit: 65536
2020-08-10 10:54:40,737 INFO snapshot.SnapshotManager: SkipList is disabled
2020-08-10 10:54:40,953 INFO util.GSet: Computing capacity for map cachedBlocks
2020-08-10 10:54:40,968 INFO util.GSet: VM type = 64-bit
2020-08-10 10:54:40,969 INFO util.GSet: 0.25% max memory 239.8 MB = 613.8 KB
2020-08-10 10:54:40,973 INFO util.GSet: capacity = 2^16 = 65536 entries
2020-08-10 10:54:41,321 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
2020-08-10 10:54:41,322 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
2020-08-10 10:54:41,322 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
2020-08-10 10:54:41,597 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
2020-08-10 10:54:41,599 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
2020-08-10 10:54:41,637 INFO util.GSet: Computing capacity for map NameNodeRetryCache
2020-08-10 10:54:41,641 INFO util.GSet: VM type = 64-bit
2020-08-10 10:54:41,644 INFO util.GSet: 0.029999999329447746% max memory 239.8 MB = 73.7 KB
2020-08-10 10:54:41,647 INFO util.GSet: capacity = 2^13 = 8192 entries
2020-08-10 10:54:42,357 INFO common.Storage: Lock on /hadoop/dfs/name/in_use.lock acquired by nodename 360@f8e90944fedf
2020-08-10 10:54:43,559 INFO namenode.FileJournalManager: Recovering unfinalized segments in /hadoop/dfs/name/current
2020-08-10 10:54:44,466 INFO namenode.FileJournalManager: Finalizing edits file /hadoop/dfs/name/current/edits_inprogress_0000000000000000032 -> /hadoop/dfs/name/current/edits_0000000000000000032-0000000000000000032
2020-08-10 10:54:45,218 INFO namenode.FSImage: Planning to load image: FSImageFile(file=/hadoop/dfs/name/current/fsimage_0000000000000000029, cpktTxId=0000000000000000029)
2020-08-10 10:54:47,482 INFO namenode.FSImageFormatPBINode: Loading 18 INodes.
2020-08-10 10:54:48,937 INFO namenode.FSImageFormatProtobuf: Loaded FSImage in 3 seconds.
2020-08-10 10:54:48,943 INFO namenode.FSImage: Loaded image for txid 29 from /hadoop/dfs/name/current/fsimage_0000000000000000029
2020-08-10 10:54:48,999 INFO namenode.FSImage: Reading org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@16c63f5 expecting start txid #30
2020-08-10 10:54:49,007 INFO namenode.FSImage: Start loading edits file /hadoop/dfs/name/current/edits_0000000000000000030-0000000000000000030 maxTxnsToRead = 9223372036854775807
2020-08-10 10:54:49,050 INFO namenode.RedundantEditLogInputStream: Fast-forwarding stream '/hadoop/dfs/name/current/edits_0000000000000000030-0000000000000000030' to transaction ID 30
2020-08-10 10:54:51,152 INFO namenode.FSImage: Loaded 1 edits file(s) (the last named /hadoop/dfs/name/current/edits_0000000000000000030-0000000000000000030) of total size 1048576.0, total edits 1.0, total load time 569.0 ms
2020-08-10 10:54:51,165 INFO namenode.RedundantEditLogInputStream: Fast-forwarding stream '/hadoop/dfs/name/current/edits_0000000000000000031-0000000000000000031' to transaction ID 30
2020-08-10 10:54:51,190 INFO namenode.RedundantEditLogInputStream: Fast-forwarding stream '/hadoop/dfs/name/current/edits_0000000000000000032-0000000000000000032' to transaction ID 30
2020-08-10 10:54:51,209 INFO namenode.FSNamesystem: Need to save fs image? false (staleImage=false, haEnabled=false, isRollingUpgrade=false)
2020-08-10 10:54:51,220 INFO namenode.FSEditLog: Starting log segment at 33
2020-08-10 10:54:53,045 INFO namenode.NameCache: initialized with 0 entries 0 lookups
2020-08-10 10:54:53,048 INFO namenode.FSNamesystem: Finished loading FSImage in 11158 msecs
2020-08-10 10:54:55,085 INFO namenode.NameNode: RPC server is binding to 0.0.0.0:9000
2020-08-10 10:54:55,728 INFO ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 1000, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
2020-08-10 10:54:55,929 INFO ipc.Server: Starting Socket Reader #1 for port 9000
2020-08-10 10:54:59,335 INFO namenode.FSNamesystem: Registered FSNamesystemState, ReplicatedBlocksState and ECBlockGroupsState MBeans.
2020-08-10 10:54:59,515 INFO namenode.LeaseManager: Number of blocks under construction: 0
2020-08-10 10:54:59,690 INFO hdfs.StateChange: STATE* Safe mode ON.
The reported blocks 0 needs additional 1 blocks to reach the threshold 0.9990 of total blocks 2.
The minimum number of live datanodes is not required. Safe mode will be turned off automatically once the thresholds have been reached.
2020-08-10 10:55:00,177 INFO ipc.Server: IPC Server Responder: starting
2020-08-10 10:55:00,209 INFO ipc.Server: IPC Server listener on 9000: starting
2020-08-10 10:55:00,265 INFO namenode.NameNode: NameNode RPC up at: namenode/172.22.0.2:9000
2020-08-10 10:55:00,330 INFO namenode.FSNamesystem: Starting services required for active state
2020-08-10 10:55:00,336 INFO namenode.FSDirectory: Initializing quota with 4 thread(s)
2020-08-10 10:55:00,709 INFO namenode.FSDirectory: Quota initialization completed in 370 milliseconds
name space=18
storage space=75
storage types=RAM_DISK=0, SSD=0, DISK=0, ARCHIVE=0, PROVIDED=0
2020-08-10 10:55:00,807 INFO blockmanagement.CacheReplicationMonitor: Starting CacheReplicationMonitor with interval 30000 milliseconds
2020-08-10 10:55:13,692 INFO hdfs.StateChange: BLOCK* registerDatanode: from DatanodeRegistration(172.22.0.5:9866, datanodeUuid=bce9d643-ec21-425e-a7b1-937090c2d19b, infoPort=9864, infoSecurePort=0, ipcPort=9867, storageInfo=lv=-57;cid=CID-eeccc7c7-041f-4051-ba9b-cc998e34b889;nsid=963490822;c=1597049905038) storage bce9d643-ec21-425e-a7b1-937090c2d19b
2020-08-10 10:55:13,706 INFO net.NetworkTopology: Adding a new node: /default-rack/172.22.0.5:9866
2020-08-10 10:55:13,708 INFO blockmanagement.BlockReportLeaseManager: Registered DN bce9d643-ec21-425e-a7b1-937090c2d19b (172.22.0.5:9866).
2020-08-10 10:55:14,185 INFO hdfs.StateChange: BLOCK* registerDatanode: from DatanodeRegistration(172.22.0.3:9866, datanodeUuid=48c6289d-41b9-4d4e-a39f-f01fc6a16d4e, infoPort=9864, infoSecurePort=0, ipcPort=9867, storageInfo=lv=-57;cid=CID-eeccc7c7-041f-4051-ba9b-cc998e34b889;nsid=963490822;c=1597049905038) storage 48c6289d-41b9-4d4e-a39f-f01fc6a16d4e
2020-08-10 10:55:14,190 INFO net.NetworkTopology: Adding a new node: /default-rack/172.22.0.3:9866
2020-08-10 10:55:14,196 INFO blockmanagement.BlockReportLeaseManager: Registered DN 48c6289d-41b9-4d4e-a39f-f01fc6a16d4e (172.22.0.3:9866).
2020-08-10 10:55:14,455 INFO hdfs.StateChange: BLOCK* registerDatanode: from DatanodeRegistration(172.22.0.4:9866, datanodeUuid=7227d6bb-7d4f-4cb7-8a1a-ee959cef4cfd, infoPort=9864, infoSecurePort=0, ipcPort=9867, storageInfo=lv=-57;cid=CID-eeccc7c7-041f-4051-ba9b-cc998e34b889;nsid=963490822;c=1597049905038) storage 7227d6bb-7d4f-4cb7-8a1a-ee959cef4cfd
2020-08-10 10:55:14,457 INFO net.NetworkTopology: Adding a new node: /default-rack/172.22.0.4:9866
2020-08-10 10:55:14,458 INFO blockmanagement.BlockReportLeaseManager: Registered DN 7227d6bb-7d4f-4cb7-8a1a-ee959cef4cfd (172.22.0.4:9866).
2020-08-10 10:55:14,786 INFO blockmanagement.DatanodeDescriptor: Adding new storage ID DS-9aa7c881-0925-4ef4-917a-2a2c9ec7b935 for DN 172.22.0.5:9866
2020-08-10 10:55:14,845 INFO blockmanagement.DatanodeDescriptor: Adding new storage ID DS-e6deaf47-6082-4c65-8b98-89edac2df8f8 for DN 172.22.0.3:9866
2020-08-10 10:55:14,991 INFO blockmanagement.DatanodeDescriptor: Adding new storage ID DS-6359f398-c37d-43eb-adc2-4978cbdff889 for DN 172.22.0.4:9866
2020-08-10 10:55:15,259 INFO BlockStateChange: BLOCK* processReport 0xf1b19f021e1b1e0f: Processing first storage report for DS-9aa7c881-0925-4ef4-917a-2a2c9ec7b935 from datanode bce9d643-ec21-425e-a7b1-937090c2d19b
2020-08-10 10:55:15,337 INFO blockmanagement.BlockManager: initializing replication queues
2020-08-10 10:55:15,344 INFO hdfs.StateChange: STATE* Safe mode extension entered.
The reported blocks 1 has reached the threshold 0.9990 of total blocks 2. The minimum number of live datanodes is not required. In safe mode extension. Safe mode will be turned off automatically in 29 seconds.
2020-08-10 10:55:15,353 INFO BlockStateChange: BLOCK* processReport 0xf1b19f021e1b1e0f: from storage DS-9aa7c881-0925-4ef4-917a-2a2c9ec7b935 node DatanodeRegistration(172.22.0.5:9866, datanodeUuid=bce9d643-ec21-425e-a7b1-937090c2d19b, infoPort=9864, infoSecurePort=0, ipcPort=9867, storageInfo=lv=-57;cid=CID-eeccc7c7-041f-4051-ba9b-cc998e34b889;nsid=963490822;c=1597049905038), blocks: 2, hasStaleStorage: false, processing time: 94 msecs, invalidatedBlocks: 0
2020-08-10 10:55:15,386 INFO blockmanagement.BlockManager: Total number of blocks = 2
2020-08-10 10:55:15,398 INFO blockmanagement.BlockManager: Number of invalid blocks = 0
2020-08-10 10:55:15,402 INFO blockmanagement.BlockManager: Number of under-replicated blocks = 1
2020-08-10 10:55:15,403 INFO blockmanagement.BlockManager: Number of over-replicated blocks = 0
2020-08-10 10:55:15,403 INFO blockmanagement.BlockManager: Number of blocks being written = 0
2020-08-10 10:55:15,403 INFO hdfs.StateChange: STATE* Replication Queue initialization scan for invalid, over- and under-replicated blocks completed in 55 msec
2020-08-10 10:55:15,404 INFO BlockStateChange: BLOCK* processReport 0x5e59b4fa3cbbe2af: Processing first storage report for DS-6359f398-c37d-43eb-adc2-4978cbdff889 from datanode 7227d6bb-7d4f-4cb7-8a1a-ee959cef4cfd
2020-08-10 10:55:15,405 INFO BlockStateChange: BLOCK* processReport 0x5e59b4fa3cbbe2af: from storage DS-6359f398-c37d-43eb-adc2-4978cbdff889 node DatanodeRegistration(172.22.0.4:9866, datanodeUuid=7227d6bb-7d4f-4cb7-8a1a-ee959cef4cfd, infoPort=9864, infoSecurePort=0, ipcPort=9867, storageInfo=lv=-57;cid=CID-eeccc7c7-041f-4051-ba9b-cc998e34b889;nsid=963490822;c=1597049905038), blocks: 2, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0
2020-08-10 10:55:15,411 INFO BlockStateChange: BLOCK* processReport 0x829ff257cefe75bb: Processing first storage report for DS-e6deaf47-6082-4c65-8b98-89edac2df8f8 from datanode 48c6289d-41b9-4d4e-a39f-f01fc6a16d4e
2020-08-10 10:55:15,448 INFO BlockStateChange: BLOCK* processReport 0x829ff257cefe75bb: from storage DS-e6deaf47-6082-4c65-8b98-89edac2df8f8 node DatanodeRegistration(172.22.0.3:9866, datanodeUuid=48c6289d-41b9-4d4e-a39f-f01fc6a16d4e, infoPort=9864, infoSecurePort=0, ipcPort=9867, storageInfo=lv=-57;cid=CID-eeccc7c7-041f-4051-ba9b-cc998e34b889;nsid=963490822;c=1597049905038), blocks: 2, hasStaleStorage: false, processing time: 37 msecs, invalidatedBlocks: 0
2020-08-10 10:55:24,291 INFO ipc.Server: IPC Server handler 0 on default port 9000, call Call#0 Retry#0 org.apache.hadoop.hdfs.protocol.ClientProtocol.mkdirs from 172.22.0.6:42424: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create directory /rmstate/FSRMStateRoot/RMDTSecretManagerRoot. Name node is in safe mode.
The reported blocks 2 has reached the threshold 0.9990 of total blocks 2. The minimum number of live datanodes is not required. In safe mode extension. Safe mode will be turned off automatically in 21 seconds. NamenodeHostName:namenode
2020-08-10 10:55:35,378 INFO hdfs.StateChange: STATE* Safe mode ON, in safe mode extension.
The reported blocks 2 has reached the threshold 0.9990 of total blocks 2. The minimum number of live datanodes is not required. In safe mode extension. Safe mode will be turned off automatically in 9 seconds.
2020-08-10 10:55:45,389 INFO hdfs.StateChange: STATE* Safe mode is OFF
2020-08-10 10:55:45,389 INFO hdfs.StateChange: STATE* Leaving safe mode after 45 secs
2020-08-10 10:55:45,390 INFO hdfs.StateChange: STATE* Network topology has 1 racks and 3 datanodes
2020-08-10 10:55:45,390 INFO hdfs.StateChange: STATE* UnderReplicatedBlocks has 0 blocks
最佳答案
你能提供 docker-compose.yml 文件吗?
但我认为您的问题与端口转发有关。
您必须在 docker-compose.yml 上启用它。
例子:
services:
hadoop:
ports:
- "9870:9870"
查看
Networking on Docker
关于docker - 容器正在运行但无法访问本地主机,我该如何解决这个问题?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/63337975/
我通过 spring ioc 编写了一些 Rest 应用程序。但我无法解决这个问题。这是我的异常(exception): org.springframework.beans.factory.BeanC
我对 TestNG、Spring 框架等完全陌生,我正在尝试使用注释 @Value通过 @Configuration 访问配置文件注释。 我在这里想要实现的目标是让控制台从配置文件中写出“hi”,通过
为此工作了几个小时。我完全被难住了。 这是 CS113 的实验室。 如果用户在程序(二进制计算器)结束时选择继续,我们需要使用 goto 语句来到达程序的顶部。 但是,我们还需要释放所有分配的内存。
我正在尝试使用 ffmpeg 库构建一个小的 C 程序。但是我什至无法使用 avformat_open_input() 打开音频文件设置检查错误代码的函数后,我得到以下输出: Error code:
使用 Spring Initializer 创建一个简单的 Spring boot。我只在可用选项下选择 DevTools。 创建项目后,无需对其进行任何更改,即可正常运行程序。 现在,当我尝试在项目
所以我只是在 Mac OS X 中通过 brew 安装了 qt。但是它无法链接它。当我尝试运行 brew link qt 或 brew link --overwrite qt 我得到以下信息: ton
我在提交和 pull 时遇到了问题:在提交的 IDE 中,我看到: warning not all local changes may be shown due to an error: unable
我跑 man gcc | grep "-L" 我明白了 Usage: grep [OPTION]... PATTERN [FILE]... Try `grep --help' for more inf
我有一段代码,旨在接收任何 URL 并将其从网络上撕下来。到目前为止,它运行良好,直到有人给了它这个 URL: http://www.aspensurgical.com/static/images/a
在过去的 5 个小时里,我一直在尝试在我的服务器上设置 WireGuard,但在完成所有设置后,我无法 ping IP 或解析域。 下面是服务器配置 [Interface] Address = 10.
我正在尝试在 GitLab 中 fork 我的一个私有(private)项目,但是当我按下 fork 按钮时,我会收到以下信息: No available namespaces to fork the
我这里遇到了一些问题。我是 node.js 和 Rest API 的新手,但我正在尝试自学。我制作了 REST API,使用 MongoDB 与我的数据库进行通信,我使用 Postman 来测试我的路
下面的代码在控制台中给出以下消息: Uncaught DOMException: Failed to execute 'appendChild' on 'Node': The new child el
我正在尝试调用一个新端点来显示数据,我意识到在上一组有效的数据中,它在数据周围用一对额外的“[]”括号进行控制台,我认为这就是问题是,而新端点不会以我使用数据的方式产生它! 这是 NgFor 失败的原
我正在尝试将我的 Symfony2 应用程序部署到我的 Azure Web 应用程序,但遇到了一些麻烦。 推送到远程时,我在终端中收到以下消息 remote: Updating branch 'mas
Minikube已启动并正在运行,没有任何错误,但是我无法 curl IP。我在这里遵循:https://docs.traefik.io/user-guide/kubernetes/,似乎没有提到关闭
每当我尝试docker组成任何项目时,都会出现以下错误。 我尝试过有和没有sudo 我在这台机器上只有这个问题。我可以在Mac和Amazon WorkSpace上运行相同的容器。 (myslabs)
我正在尝试 pip install stanza 并收到此消息: ERROR: No matching distribution found for torch>=1.3.0 (from stanza
DNS 解析看起来不错,但我无法 ping 我的服务。可能是什么原因? 来自集群中的另一个 Pod: $ ping backend PING backend.default.svc.cluster.l
我正在使用Hibernate 4 + Spring MVC 4当我开始 Apache Tomcat Server 8我收到此错误: Error creating bean with name 'wel
我是一名优秀的程序员,十分优秀!