- android - 多次调用 OnPrimaryClipChangedListener
- android - 无法更新 RecyclerView 中的 TextView 字段
- android.database.CursorIndexOutOfBoundsException : Index 0 requested, 光标大小为 0
- android - 使用 AppCompat 时,我们是否需要明确指定其 UI 组件(Spinner、EditText)颜色
我能够从一个空的数据存储自动生成一个 bulkloader.yaml
文件。它只包含 python_preamble
,但 transformers
部分是空的。
python_preamble:
- import: google.appengine.ext.bulkload.transform
- import: google.appengine.ext.bulkload.bulkloader_wizard
- import: my_own_transformers
- import: data_models # This is where the SomeData class is defined.
# some more imports here
然后根据文档中的示例,我需要为我的 CSV 中的每一列定义一个属性映射:
transformers:
- kind: SomeData
connector: csv
property_map:
- property: date
import_transform: transform.some_undocumented_function
我的理解是定义为import_transform
的函数会将普通的CSV 字符串转换为Property Class。值得数据存储。我想了解转换的工作原理,所以我认为我有两种选择。
google.appengine.ext.bulkload.transform
的库引用在哪里?我想知道如何使用 transform.some_undocumented_function
,以及所有其他 transform.some_other_undocumented_transformers
您可以从我的 python_preamble
中看到我 - import: my_own_transformers
。在该模块中,我定义了一个函数 transform_date
,它采用 ISO 日期字符串(例如 2001-01-01
)并将其转换为适合 db 的类型.DateProperty()
。如果我的概念是正确的,我可以使用:
property_map: - property: date import_transform: my_own_transforms.transform_date
最佳答案
1)
您可以查看 source code或者给交互式控制台这样的东西:
from google.appengine.ext.bulkload import transform
help(transform)
你会得到:
Help on module google.appengine.ext.bulkload.transform in google.appengine.ext.bulkload:
NAME
google.appengine.ext.bulkload.transform - Bulkloader Transform Helper functions.
FILE
/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/bulkload/transform.py
DESCRIPTION
A collection of helper functions for bulkloading data, typically referenced
from a bulkloader.yaml file.
FUNCTIONS
blob_to_file(filename_hint_propertyname=None, directory_hint='')
Write the blob contents to a file, and replace them with the filename.
Args:
filename_hint_propertyname: If present, the filename will begin with
the contents of this value in the entity being exported.
directory_hint: If present, the files will be stored in this directory.
Returns:
A function which writes the input blob to a file.
blobproperty_from_base64 = wrapper(value)
bytestring_from_base64 = wrapper(value)
child_node_from_list(child_node_name)
Return a value suitable for generating an XML child node on export.
The return value is a list of tuples which the simplexml connector will
use to build a child node.
See also list_from_child_node
Args:
child_node_name: The name to use for each child node.
Returns:
Transform function which works as described in the args.
create_deep_key(*path_info)
A method to make multi-level Key objects.
Generates multi-level key from multiple fields in the input dictionary.
This is typically used for Keys for entities which have variable parent keys,
e.g. ones with owned relationships. It can used for both __key__ and
references.
Use create_foreign_key as a simpler way to create single level keys.
Args:
path_info: List of tuples, describing (kind, property, is_id=False).
kind: The kind name.
property: The external property in the current import dictionary, or
transform.CURRENT_PROPERTY for the value passed to the transform.
is_id: Converts value to int and treats as numeric ID if True, otherwise
the value is a string name. Default is False.
Example:
create_deep_key(('rootkind', 'rootcolumn'),
('childkind', 'childcolumn', True),
('leafkind', transform.CURRENT_PROPERTY))
Returns:
Transform method which parses the info from the current neutral dictionary
into a Key with parents as described by path_info.
create_foreign_key(kind, key_is_id=False)
A method to make one-level Key objects.
These are typically used in ReferenceProperty in Python, where the reference
value is a key with kind (or model) name name.
This helper method does not support keys with parents. Use create_deep_key
instead to create keys with parents.
Args:
kind: The kind name of the reference as a string.
key_is_id: If true, convert the key into an integer to be used as an id.
If false, leave the key in the input format (typically a string).
Returns:
Single argument method which parses a value into a Key of kind entity_kind.
empty_if_none(fn)
A wrapper for a value to return '' if it's None. Useful on export.
Can be used in config files (e.g. "transform.empty_if_none(unicode)" or
as a decorator.
Args:
fn: Single argument transform function.
Returns:
Wrapped function.
export_date_time(format)
A wrapper around strftime. Also returns '' if the input is None.
Args:
format: Format string for strftime.
Returns:
Single argument method which convers a datetime into a string using format.
import_date_time(format, _strptime=None)
A wrapper around strptime. Also returns None if the input is empty.
Args:
format: Format string for strptime.
Returns:
Single argument method which parses a string into a datetime using format.
join_list(delimeter)
Join a list into a string using the delimeter.
This is just a wrapper for string.join.
Args:
delimeter: The delimiter to use when joining the string.
Returns:
Method which joins the list into a string with the delimeter.
key_id_or_name_as_string = transform_function(key)
key_id_or_name_as_string_n(index)
Pull out the nth (0-based) key id or name from a key which has parents.
If a key is present, return its id or name as a string.
Note that this loses the distinction between integer IDs and strings
which happen to look like integers. Use key_type to distinguish them.
This is a useful complement to create_deep_key.
Args:
index: The depth of the id or name to extract. Zero is the root key.
Negative one is the leaf key.
Returns:
Function extracting the name or ID of the key at depth index, as a unicode
string. Returns '' if key is empty (unsaved), otherwise raises IndexError
if the key is not as deep as described.
key_kind = wrapper(value)
key_kind_n(index)
Pull out the nth (0-based) key kind from a key which has parents.
This is a useful complement to create_deep_key.
Args:
index: The depth of the id or name to extract. Zero is the root key.
Negative one is the leaf key.
Returns:
Function returning the kind of the key at depth index, or raising
IndexError if the key is not as deep as described.
key_type = transform_function(key)
key_type_n(index)
Pull out the nth (0-based) key type from a key which has parents.
This is most useful when paired with key_id_or_name_as_string_n.
This is a useful complement to create_deep_key.
Args:
index: The depth of the id or name to extract. Zero is the root key.
Negative one is the leaf key.
Returns:
Method returning the type ('ID' or 'name') of the key at depth index.
Returns '' if key is empty (unsaved), otherwise raises IndexError
if the key is not as deep as described.
list_from_child_node(xpath, suppress_blank=False)
Return a list property from child nodes of the current xml node.
This applies only the simplexml helper, as it assumes __node__, the current
ElementTree node corresponding to the import record.
Sample usage for structure:
<Visit>
<VisitActivities>
<Activity>A1</Activity>
<Activity>A2</Activity>
</VisitActivities>
</Visit>
property: activities
external_name: VisitActivities # Ignored on import, used on export.
import_transform: list_from_xml_node('VisitActivities/Activity')
export_transform: child_node_from_list('Activity')
Args:
xpath: XPath to run on the current node.
suppress_blank: if True, ndoes with no text will be skipped.
Returns:
Transform function which works as described in the args.
list_from_multiproperty(*external_names)
Create a list from multiple properties.
Args:
external_names: List of the properties to use.
Returns:
Transform function which returns a list of the properties in external_names.
none_if_empty(fn)
A decorator which returns None if its input is empty else fn(x).
Useful on import. Can be used in config files
(e.g. "transform.none_if_empty(int)" or as a decorator.
Args:
fn: Single argument transform function.
Returns:
Wrapped function.
property_from_list(index)
Return the Nth item from a list, or '' if the list is shorter.
Args:
index: Item in the list to return.
Returns:
Function returning the item from a list, or '' if the list is too short.
regexp_bool(regexp, flags=0)
Return a boolean if the expression matches with re.match.
Note that re.match anchors at the start but not end of the string.
Args:
regexp: String, regular expression.
flags: Optional flags to pass to re.match.
Returns:
Method which returns a Boolean if the expression matches.
regexp_extract(pattern, method=<function match at 0x336270>, group=1)
Return first group in the value matching the pattern using re.match.
Args:
pattern: A regular expression to match on with at least one group.
method: The method to use for matching; normally re.match or re.search.
group: The group to use for extracting a value.
Returns:
A single argument method which returns the group_arg group matched,
or None if no match was found or the input was empty.
regexp_to_list(pattern)
Return function that returns a list of objects that match the regex.
Useful on import. Uses the provided regex to split a string value into a list
of strings. Wrapped by none_if_input_or_result_empty, so returns none if
there are no matches for the regex and none if the input is empty.
Args:
pattern: A regular expression pattern to match against the input string.
Returns:
None if the input was none or no matches were found, otherwise a list of
strings matching the input expression.
split_string(delimeter)
Split a string using the delimeter into a list.
This is just a wrapper for string.split.
Args:
delimeter: The delimiter to split the string on.
Returns:
Method which splits the string into a list along the delimeter.
DATA
CURRENT_PROPERTY = None
KEY_TYPE_ID = 'ID'
KEY_TYPE_NAME = 'name'
__loader__ = <google.appengine.tools.dev_appserver.HardenedModulesHook...
2)
确切地说,您可以使用您定义的转换函数,或者在这种特定情况下,您可以直接使用 transform.import_date_time
。
import_date_time(format, _strptime=None)
A wrapper around strptime. Also returns None if the input is empty.
Args:
format: Format string for strptime.
Returns:
Single argument method which parses a string into a datetime using format.
关于google-app-engine - Google App Engine bulkloader 转换的引用页在哪里?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/6817626/
我应该如何在配置 yaml 文件中配置导入转换和导出转换,以便能够导出和导入多个整数属性? 最佳答案 我编写了这个用于批量加载数据的助手集合: http://code.google.com/p/bul
简而言之:如何配置 bulkloader 以将数据插入到 2 个带有引用的模型中? 我有一个 person 和 fruit 类,person 链接到 fruit: class Fruit(db.Mod
我正在尝试将 C# 代码中的 SQLXMLBulkLoader4 用于 SQL 2008 数据库。但由于某种原因,尽管没有抛出任何错误,但它根本没有插入任何行。我已经使用了 bulkloads 自己的
我想使用 bulkloader 下载具有一些自定义属性的模型中的所有实体。 如果我这样定义一个模型, class MyType: def __init__(self, arg):
我正在做一些应该很简单的事情,却一遍又一遍地卡住了。将非常简单的 csv 文件上传到我的开发服务器时出现此错误: Error in WorkerThread-0: app "dev~fbdec" ca
我目前正在尝试从我继承的 Google 应用下载和导出表格。我正在使用他们的 Python 框架。以下是 bulkloader.yaml 文件中的相关行: - kind: KindName con
在默认的 bulkloader.yaml 中,它说: # - Check that __key__ property is what you want. Its value will becom
我们将Elasticsearch集群从2.1.1升级到2.2。我们通过2.2.1之前版本运行的bulkload处理器进行的批量加载过程现在引发以下异常。如果我想念什么,请告诉我。我是这个新手。 [ma
我正在尝试使用 Bulkloader 从我的数据存储下载数据,但 DateTimeProperty 的精度似乎不准确,除非我以错误的方式做事。 这是我要下载的数据的属性: class LoggingP
我正在使用 MYSqlBulkLoader 将文本文件加载到 mysql 数据库表中。但是,文件中的日期格式为 mm/dd/yyyy,而 MySql 期望日期格式为 yyyy-mm-dd。用正确的格式
谁能澄清一下这里的情况? 我已将 Bulkloader 与 Python25 和旧/主从数据库一起使用。 我也将它(仅用于上传数据)与 Python25 和高复制数据存储一起使用,尽管它在这里说: h
我在使用 UWP 应用程序将数据从 csv 加载到 mysql 数据库时遇到问题。我基本上想做的是: 通过 HttpClient 从 csv 文件读取数据(200k 条记录,32 MB) 将数据保存到
我有相当多的数据要上传到 Google App Engine。我想使用 bulkloader 来帮助将其放入其中。但是,我的数据太多,通常在完成之前用完我的 CPU 配额。此外,任何其他问题(如互联网
我正在尝试使用 bulkloader 填充 db.ListProperty() 模型字段。 我正在使用如下导入转换函数: def parse_array(fn): def wrapper(va
我之前已经在我的项目中成功使用了bulkloader,但最近我在修改记录时添加了一个新字段来表示时间戳。不过,这个新字段给我带来了麻烦,因为它默认为空。在导入之前,如果没有在 csv 中手动插入时间戳
如何将上次修改时间属性添加到在批量上传期间更新的实体类型? 我目前正在使用 appcfg upload_data 将 csv 上传到我的高复制数据存储。我计划将其作为一项 cron 作业,以进行从内部
在AppEngine Bulkloader Documentation ,他们声明“如果您尝试从 High Replication 数据存储复制,您将在管理控制台中看到 high_replicatio
我想问一下,使用 Google App Engine BulkLoader 类导入数据需要提供哪种凭证 appcfg.py upload_data --config_file=models.py --
我正在尝试使用以下方法从数据存储中下载一些数据命令: appcfg.py download_data --config_file=bulkloader.yaml --application=myapp
对于我的项目,我使用 GAE 数据存储来存储数据。用于备份目的我决定使用 bulkloader - 它下载整个数据完美地保存在 csv 文件中。上传也上传数据很好没有错误。 我的问题是,上传不更新现有
我是一名优秀的程序员,十分优秀!