- android - 多次调用 OnPrimaryClipChangedListener
- android - 无法更新 RecyclerView 中的 TextView 字段
- android.database.CursorIndexOutOfBoundsException : Index 0 requested, 光标大小为 0
- android - 使用 AppCompat 时,我们是否需要明确指定其 UI 组件(Spinner、EditText)颜色
我正在尝试使用此处所述的 requirements.txt 文件运行我的 Cloud DataFlow 作业
https://cloud.google.com/dataflow/pipelines/dependencies-python
与其从源代码构建所有 opencv(需要 20-30 分钟),我可以只构建 python 库
通过我的计算引擎,我可以做到这一 pip
root@fcfca6a4dad2:/DeepMeerkat# pip install opencv-python
Collecting opencv-python
Downloading opencv_python-3.2.0.7-cp27-cp27mu-manylinux1_x86_64.whl (6.7MB)
100% |################################| 6.7MB 163kB/s
Collecting numpy>=1.11.1 (from opencv-python)
Downloading numpy-1.13.0-cp27-cp27mu-manylinux1_x86_64.whl (16.6MB)
100% |################################| 16.6MB 68kB/s
Installing collected packages: numpy, opencv-python
Found existing installation: numpy 1.8.2
DEPRECATION: Uninstalling a distutils installed project (numpy) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
Uninstalling numpy-1.8.2:
Successfully uninstalled numpy-1.8.2
Successfully installed numpy-1.13.0 opencv-python-3.2.0.7
我可以将其与其他一些模块一起包装到一个需求文件中
root@fcfca6a4dad2:/DeepMeerkat# pip install -r tests/prediction/requirements.txt
Requirement already satisfied: opencv-python in /usr/local/lib/python2.7/dist-packages (from -r tests/prediction/requirements.txt (line 1))
Collecting tensorflow==1.0.1 (from -r tests/prediction/requirements.txt (line 2))
Downloading tensorflow-1.0.1-cp27-cp27mu-manylinux1_x86_64.whl (44.1MB)
100% |################################| 44.1MB 27kB/s
Requirement already satisfied: numpy in /usr/local/lib/python2.7/dist-packages (from -r tests/prediction/requirements.txt (line 3))
Requirement already satisfied: mock>=2.0.0 in /usr/local/lib/python2.7/dist-packages (from tensorflow==1.0.1->-r tests/prediction/requirements.txt (line 2))
Requirement already satisfied: wheel in /usr/lib/python2.7/dist-packages (from tensorflow==1.0.1->-r tests/prediction/requirements.txt (line 2))
Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python2.7/dist-packages (from tensorflow==1.0.1->-r tests/prediction/requirements.txt (line 2))
Requirement already satisfied: protobuf>=3.1.0 in /usr/local/lib/python2.7/dist-packages (from tensorflow==1.0.1->-r tests/prediction/requirements.txt (line 2))
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /usr/local/lib/python2.7/dist-packages (from mock>=2.0.0->tensorflow==1.0.1->-r tests/prediction/requirements.txt (line 2))
Requirement already satisfied: pbr>=0.11 in /usr/local/lib/python2.7/dist-packages (from mock>=2.0.0->tensorflow==1.0.1->-r tests/prediction/requirements.txt (line 2))
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf>=3.1.0->tensorflow==1.0.1->-r tests/prediction/requirements.txt (line 2))
Installing collected packages: tensorflow
Successfully installed tensorflow-1.0.1
但是,当我将它发送到云数据流时,它无法从 worker 中找到 opencv-python。
root@fcfca6a4dad2:/DeepMeerkat# python tests/prediction/run.py \
> --runner DataflowRunner \
> --project $PROJECT \
> --staging_location $BUCKET/staging \
> --temp_location $BUCKET/temp \
> --job_name $PROJECT-deepmeerkat \
> --setup_file tests/prediction/setup.py \
> --requirements_file tests/prediction/requirements.txt
No handlers could be found for logger "oauth2client.contrib.multistore_file"
/usr/local/lib/python2.7/dist-packages/apache_beam/io/gcp/gcsio.py:113: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
INFO:root:Starting the size estimation of the input
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:root:Finished the size estimation of the input at 1 files. Estimation took 0.0855119228363 seconds
INFO:root:Starting the size estimation of the input
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:root:Finished the size estimation of the input at 1 files. Estimation took 0.0597159862518 seconds
/usr/local/lib/python2.7/dist-packages/apache_beam/coders/typecoders.py:135: UserWarning: Using fallback coder for typehint: Any.
warnings.warn('Using fallback coder for typehint: %r.' % typehint)
INFO:root:Starting GCS upload to gs://api-project-773889352370-testing/staging/api-project-773889352370-deepmeerkat.1499372970.163850/requirements.txt...
INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
INFO:root:Completed GCS upload to gs://api-project-773889352370-testing/staging/api-project-773889352370-deepmeerkat.1499372970.163850/requirements.txt
INFO:root:Executing command: ['/usr/bin/python', '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'tests/prediction/requirements.txt', '--no-binary', ':all:']
DEPRECATION: pip install --download has been deprecated and will be removed in the future. Pip now has a download command that should be used instead.
Collecting opencv-python (from -r tests/prediction/requirements.txt (line 1))
Could not find a version that satisfies the requirement opencv-python (from -r tests/prediction/requirements.txt (line 1)) (from versions: )
No matching distribution found for opencv-python (from -r tests/prediction/requirements.txt (line 1))
Traceback (most recent call last):
File "tests/prediction/run.py", line 22, in <module>
predict.run()
File "/DeepMeerkat/tests/prediction/modules/predict.py", line 32, in run
p.run()
File "/usr/local/lib/python2.7/dist-packages/apache_beam/pipeline.py", line 167, in run
self.to_runner_api(), self.runner, self._options).run(False)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/pipeline.py", line 176, in run
return self.runner.run(self)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 252, in run
self.dataflow_client.create_job(self.job), self)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/utils/retry.py", line 168, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/apiclient.py", line 425, in create_job
self.create_job_description(job)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/apiclient.py", line 448, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/dependency.py", line 307, in stage_job_resources
setup_options.requirements_file, requirements_cache_path)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/dependency.py", line 241, in _populate_requirements_cache
processes.check_call(cmd_args)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/utils/processes.py", line 44, in check_call
return subprocess.check_call(*args, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python', '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'tests/prediction/requirements.txt', '--no-binary', ':all:']' returned non-zero exit status 1
问题似乎出在没有二进制标记上。本地运行(以上卸载后)
root@fcfca6a4dad2:/DeepMeerkat# pip install -r tests/prediction/requirements.txt --no-binary :all:
Collecting opencv-python (from -r tests/prediction/requirements.txt (line 1))
Could not find a version that satisfies the requirement opencv-python (from -r tests/prediction/requirements.txt (line 1)) (from versions: )
No matching distribution found for opencv-python (from -r tests/prediction/requirements.txt (line 1))
no-binary 标志被描述为排除破轮?这在这种情况下如何适用?
可以确认模块可以运行
再次,
root@fcfca6a4dad2:/DeepMeerkat# pip install opencv-python
Collecting opencv-python
Using cached opencv_python-3.2.0.7-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: numpy>=1.11.1 in /usr/local/lib/python2.7/dist-packages (from opencv-python)
Installing collected packages: opencv-python
Successfully installed opencv-python-3.2.0.7
root@fcfca6a4dad2:/DeepMeerkat# python
Python 2.7.9 (default, Jun 29 2016, 13:08:31)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>>
最佳答案
我认为您看到的错误实际上是由于工作人员未能安装 wheel 文件而引起的。如 opencv-python package page 中所述wheel 文件的问题可能会导致包显示为未找到。
在这种情况下,您可以使用非 PyPI 包的说明并指定 --extra_package <local path to wheel file>
而不是将 opencv-python 添加为要求。这应该会导致 wheel 文件在每个 worker 中暂存和安装。
关于Cloud DataFlow 中的 Python 依赖项,requirements.txt 在本地工作,但在 worker 上不工作,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/44958387/
我知道我要求太多,但也许你也可以帮助解决这个问题。 a.txt 包含单词,b.txt 包含字符串。 我想知道 b.txt 中有多少个字符串以 a.txt 中的单词结尾 例子:一个.txt apple
这个问题在这里已经有了答案: erge text files ordered by numerical filenames in Bash (3 个答案) 关闭 4 年前。 我有一个文件夹,其中包含
我在一个目录中有几个平面文件 (.txt)。所有这些文件的格式都是 *.txt.txt,所以我想将其重命名为 *.txt?有什么简单的方法可以一起重命名? 当我尝试 ren *.txt.txt *.t
这个问题在这里已经有了答案: How can I use a file in a command and redirect output to the same file without trunc
您是否有任何理由应该或不应该允许访问 javascript 或 css 文件?特别是常见的文件,如 jquery。 最佳答案 人们普遍认为,搜索引擎每天为给定站点分配一定数量的带宽或 URL。因此,一
Closed. This question is off-topic。它当前不接受答案。 想要改善这个问题吗? Update the question,所以它是用于堆栈溢出的on-topic。 已关闭
这是相同的代码。我面临的问题是,我无法在任何文件上写入任何内容。请帮忙解决这个问题 #include #include int main() { FILE *fe; FILE *fo;
我想要特定于域的 robots.txt,到目前为止这有效: RewriteRule ^robots\.txt$ robots/%{HTTP_HOST}.txt [L] 但我希望有一个后备方案,因此如果
我正在调试一些构建成功运行的 SQL 命令的代码。 然而,在查询结束时,查询结果似乎被写入了一个文本文件。 完整的查询如下 echo SELECT DATE,DATETABLE,DATE,APPDAT
这个问题已经有答案了: difference between grep Vs cat and grep (5 个回答) 已关闭 8 年前。 我看到一个例子,其中有人这样做: cat source.tx
我想将表中的数据从以 csv 格式存储的文本文件插入到 sql server 表中。为此,我正在使用批量插入语句。现在我需要在“From”子句中指定文件名。我不想在那里使用网络位置或本地位置。我想将我
假设我有一个测试文件夹 (test.domain.com) 并且我不希望搜索引擎在其中抓取,我是否需要在测试文件夹中有一个 robots.txt 或者我可以只放置一个 robots.txt在根目录中,
关闭。这个问题是off-topic .它目前不接受答案。 想改善这个问题吗? Update the question所以它是 on-topic对于堆栈溢出。 9年前关闭。 Improve this q
这个问题在这里已经有了答案: order of directives in robots.txt, do they overwrite each other or complement each ot
关闭。这个问题是opinion-based .它目前不接受答案。 想改进这个问题?更新问题,以便 editing this post 可以用事实和引用来回答它. 8年前关闭。 Improve this
已关闭。这个问题是 not about programming or software development 。目前不接受答案。 这个问题似乎不是关于 a specific programming
在过去的几年中,当我引用“名字”字段的文本框控件时,我一直使用 FirstNameTxt 命名约定。但是,我注意到大多数其他开发人员倾向于使用命名约定 txtFirstName 哪个是最好的约定?为什
我只想允许目录 /minsc 中的一个文件,但我想禁止该目录的其余部分。 现在 robots.txt 中是这样的: User-agent: * Crawl-delay: 10 # Directorie
我正在编写一个将 youtube.com 映射到另一个域的代理服务器(因此用户可以轻松地从德国等国家/地区访问 youtube,而无需审查搜索结果和视频)。 不幸的是,我的 robots.txt 中存
我没有编程技能,但有一项非常具体的任务:我必须将一个庞大的文本文件拆分成多个,并在特定的文本标记 (@) 处拆分它们。我决定尝试使用 Powershell 脚本来完成此任务。 到目前为止,这就是我想出
我是一名优秀的程序员,十分优秀!