gpt4 book ai didi

python - 使用 SQLAlchemy 和 MySQL 进行浮点值舍入

转载 作者:行者123 更新时间:2023-11-29 06:38:26 24 4
gpt4 key购买 nike

我的问题类似于这个未回答的问题:SQLAlchemy commits makes float to be rounded

我有一个如下所示的数据文本文件:

#file camera date mjd focus error
ibcy02blq UVIS1 08/03/09 55046.196630 0.57857 0.55440
ibcy02bnq UVIS1 08/03/09 55046.198330 -0.15000 0.42111
ibcy03j8q UVIS1 08/11/09 55054.041650 -0.37143 0.40802
ibcy03jaq UVIS1 08/11/09 55054.043350 -0.91857 0.51859
ibcy04m4q UVIS1 08/18/09 55061.154900 -0.32333 0.52327
ibcy04m6q UVIS1 08/18/09 55061.156600 -0.24867 0.66651
ibcy05b7q UVIS1 09/05/09 55079.912670 0.64900 0.58423
ibcy05b9q UVIS1 09/05/09 55079.914370 0.82000 0.50202
ibcy06meq UVIS1 10/02/09 55106.909840 -0.09667 0.24016

但是当我将它读入我的 MySQL 数据库后,它看起来像这样:

+------+-----------+--------+------------+---------+----------+
| id | filename | camera | date | mjd | focus |
+------+-----------+--------+------------+---------+----------+
| 1026 | ibcy02blq | UVIS1 | 2009-08-03 | 55046.2 | 0.57857 |
| 1027 | ibcy02bnq | UVIS1 | 2009-08-03 | 55046.2 | -0.15 |
| 1028 | ibcy03j8q | UVIS1 | 2009-08-11 | 55054 | -0.37143 |
| 1029 | ibcy03jaq | UVIS1 | 2009-08-11 | 55054 | -0.91857 |
| 1030 | ibcy04m4q | UVIS1 | 2009-08-18 | 55061.2 | -0.32333 |
| 1031 | ibcy04m6q | UVIS1 | 2009-08-18 | 55061.2 | -0.24867 |
| 1032 | ibcy05b7q | UVIS1 | 2009-09-05 | 55079.9 | 0.649 |
| 1033 | ibcy05b9q | UVIS1 | 2009-09-05 | 55079.9 | 0.82 |
| 1034 | ibcy06meq | UVIS1 | 2009-10-02 | 55106.9 | -0.09667 |
| 1035 | ibcy06mgq | UVIS1 | 2009-10-02 | 55106.9 | -0.1425 |
+------+-----------+--------+------------+---------+----------+

mjd 列被截断了,我不确定为什么。我知道 1/3 之类的东西存在浮点精度错误,但这看起来更像是正在实现某种类型的舍入。

这是我用来将数据提取到数据库中的代码:

def make_focus_table_main():
"""The main controller for the make_focus_table
module."""
logging.info('Process Starting')
filename_list = glob.glob('/grp/hst/OTA/focus/source/FocusModel/UVIS*FocusHistory.txt')
logging.info('Found {} files'.format(len(filename_list)))
for filename in filename_list:
logging.info('Reading data from {}'.format(filename))
output_list = []
with open(filename, 'r') as f:
data = f.readlines()
for line in data[1:]:
line = line.split()
output_dict = {}
output_dict['filename'] = line[0]
output_dict['camera'] = line[1]
output_dict['date'] = datetime.strptime(line[2], '%m/%d/%y')
output_dict['mjd'] = float(line[3])
output_dict['focus'] = float(line[4])
output_list.append(output_dict)
logging.info('Beginning bulk insert of records.')
engine.execute(Focus.__table__.insert(), output_list)
logging.info('Database insert complete.')
logging.info('Process Complete')

我已经使用 pdb 检查值在传递到数据库之前是否未被截断(即 Python/SQLAlchemy 未执行舍入)。我可以在 INSERT 命令中验证这一点 SQLAlchemy 问题:

2014-04-11 13:08:20,522 INFO sqlalchemy.engine.base.Engine INSERT INTO focus (filename, camera, date, mjd, focus) VALUES (%s, %s, %s, %s, %s)
2014-04-11 13:08:20,602 INFO sqlalchemy.engine.base.Engine (
('ibcy02blq', 'UVIS2', datetime.datetime(2009, 8, 3, 0, 0), 55046.19663, 1.05778),
('ibcy02bnq', 'UVIS2', datetime.datetime(2009, 8, 3, 0, 0), 55046.19833, 1.32333),
('ibcy03j8q', 'UVIS2', datetime.datetime(2009, 8, 11, 0, 0), 55054.04165, 1.57333),
('ibcy03jaq', 'UVIS2', datetime.datetime(2009, 8, 11, 0, 0), 55054.04335, 0.54333),
('ibcy04m4q', 'UVIS2', datetime.datetime(2009, 8, 18, 0, 0), 55061.1549, -1.152),
('ibcy04m6q', 'UVIS2', datetime.datetime(2009, 8, 18, 0, 0), 55061.1566, -1.20733),
('ibcy05b7q', 'UVIS2', datetime.datetime(2009, 9, 5, 0, 0), 55079.91267, 2.35905),
('ibcy05b9q', 'UVIS2', datetime.datetime(2009, 9, 5, 0, 0), 55079.91437, 1.84524)
... displaying 10 of 1025 total bound parameter sets ...
('ichl05qwq', 'UVIS2', datetime.datetime(2014, 4, 2, 0, 0), 56749.05103, -2.98),
('ichl05qxq', 'UVIS2', datetime.datetime(2014, 4, 2, 0, 0), 56749.05177, -3.07))
2014-04-11 13:08:20,959 INFO sqlalchemy.engine.base.Engine COMMIT

下面是列在我的 SQLAlchemy 类中的定义方式:

class Focus(Base):
"""ORM for the table storing the focus measurement information."""
__tablename__ = 'focus'
id = Column(Integer(), primary_key=True)
filename = Column(String(17), index=True, nullable=False)
camera = Column(String(5), index=True, nullable=False)
date = Column(Date(), index=True, nullable=False)
mjd = Column(Float(precision=20, scale=10), index=True, nullable=False)
focus = Column(Float(15), nullable=False)
__table_args__ = (UniqueConstraint('filename', 'camera',
name='focus_uniqueness_constraint'),)

这是在我创建表时使用 echo=True 从 SQLAlchemy 记录的 SQL:

CREATE TABLE focus (
id INTEGER NOT NULL AUTO_INCREMENT,
filename VARCHAR(17) NOT NULL,
camera VARCHAR(5) NOT NULL,
date DATE NOT NULL,
mjd FLOAT(20) NOT NULL,
focus FLOAT(15) NOT NULL,
PRIMARY KEY (id),
CONSTRAINT focus_uniqueness_constraint UNIQUE (filename, camera)
)

到目前为止,还不错。但这是我看到的带有 SHOW CREATE TABLE focus; 的 MySQL:

CREATE TABLE `focus` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`filename` varchar(17) NOT NULL,
`camera` varchar(5) NOT NULL,
`date` date NOT NULL,
`mjd` float NOT NULL,
`focus` float NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `focus_uniqueness_constraint` (`filename`,`camera`),
KEY `ix_focus_filename` (`filename`),
KEY `ix_focus_mjd` (`mjd`),
KEY `ix_focus_date` (`date`),
KEY `ix_focus_camera` (`camera`)
) ENGINE=InnoDB AUTO_INCREMENT=1193 DEFAULT CHARSET=latin1

FLOAT 定义不知何故发生了变化!这是某种类型的 MySQL 配置设置吗?我现在只是在我的本地主机上运行它,但如果这是一个配置设置,我担心如果我继续使用 float ,这段代码在生产服务器上的可移植性。我可以像在其他 SO 问题中看到的那样切换到小数列类型,因为我需要精确的值,但我想了解这里发生了什么。


更新:只是为了稍微扩展一下两位炼金术士的回答,这里是它如何改变我的查询:

> SELECT ROUND(mjd,10) FROM focus LIMIT 10;
+------------------+
| ROUND(mjd,10) |
+------------------+
| 55046.1953125000 |
| 55046.1992187500 |
| 55054.0429687500 |
| 55054.0429687500 |
| 55061.1562500000 |
| 55061.1562500000 |
| 55079.9140625000 |
| 55079.9140625000 |
| 55106.9101562500 |
| 55106.9101562500 |
+------------------+
10 rows in set (0.00 sec)

请注意,所有小数精度仍然存在。我不知道 SELECT 是舍入值,但如果您考虑浮点表示的工作原理,我想这是有道理的。它使用为该数字分配的完整字节,您显示的小数位数是任意的,直到 float 的全长:https://stackoverflow.com/a/20482699/1216837

指定精度似乎只影响它是存储为 double 还是单精度:http://dev.mysql.com/doc/refman/5.0/en/floating-point-types.html .

但是,有趣/烦人的是,当从 SQLAlchemy 层发出 SELECT 时,我不得不担心同样的事情:

query = psf_session.query(Focus).first()
print query.filename, query.mjd, query.focus

给我 bcy02blq 55046.2 1.05778 所以值仍在四舍五入。同样,这是有道理的,因为 SQLAlchemy 只是发出 SQL 命令。总而言之,这促使我切换到 DECIMAL 列类型:http://dev.mysql.com/doc/refman/5.0/en/fixed-point-types.html

最佳答案

看起来您的所有值都恰好用六位数字打印(除了 .0 在几个地方被遗漏的地方)。虽然我找不到任何关于此的文档,但我怀疑这只是在 SELECT 语句的上下文中显示 float 值的默认 MySQL 行为。

根据您提供的 CREATE TABLE 语句,内部表示是正确的,因此您只需在语句中添加类似 ROUND(mjd, 3) 的内容,使用第一个参数是要四舍五入的字段,最后一个参数是要四舍五入的位数(可以比现在显示的)。

关于python - 使用 SQLAlchemy 和 MySQL 进行浮点值舍入,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/23019677/

24 4 0
Copyright 2021 - 2024 cfsdn All Rights Reserved 蜀ICP备2022000587号
广告合作:1813099741@qq.com 6ren.com