python genfromtxt问题

Question

我是 Python 的新手...这是我的问题。对于我在 Python 中测试的优化子例程，我需要用数字解析一个 csv 文件。

csv文件的格式是这样的:

Support load summary for anchor at node 5,

Load combination,FX (N),FY (N),FZ (N),MX (Nm),MY (Nm),MZ (Nm),,

Sustained,-3,-2679,120,2012,164,69,,
Operating1,1472,2710,-672,-4520,8743,-2047,,
Maximum,1472,2710,120,2012,8743,69,,
Minimum,-3,-2679,-672,-4520,164,-2047,,

Support load summary for anchor at node 40,

Load combination,FX (N),FY (N),FZ (N),MX (Nm),MY (Nm),MZ (Nm),,

Sustained,9,-3872,-196,-91,854,-3914,,
Operating1,-2027,-8027,3834,-7573,-9102,-6323,,
Maximum,9,-3872,3834,-91,854,-3914,,
Minimum,-2027,-8027,-196,-7573,-9102,-6323,,

Support load summary for anchor at node 125,

Load combination,FX (N),FY (N),FZ (N),MX (Nm),MY (Nm),MZ (Nm),,

Sustained,-7,-2448,76,264,83,1320,,
Operating1,556,-3771,-3162,-6948,-1367,1272,,
Maximum,556,-2448,76,264,83,1320,,
Minimum,-7,-3771,-3162,-6948,-1367,1272,,

Support load summary for Hanger at node 10,

Load combination,Load (N),,

Sustained,-3668,,
Operating1,-13876,,
Maximum,-3668,,
Minimum,-13876,,

Support load summary for Hanger at node 20B,

Load combination,Load (N),,

Sustained,-14305,,
Operating1,-13359,,
Maximum,-13359,,
Minimum,-14305,,

Support load summary for restraint at node 115B,

Load combination,FX (N),FY (N),FZ (N),,

Sustained,,-5655,,,
Operating1,3696,,
Maximum,,3696,,,
Minimum,,-5655,,,

我的代码主要用于以

开头的行

Operating1,
Maximum, 
Minimum,

工作(成本函数)是对这些关键字之一之后的所有数字进行总计(代数计算)。有时，正如您在上面的数据文件中看到的那样，第二列或第三列中只有一个数字。 (见数据文件末尾)，有时，根本没有数字，如以下文件片段(见下面的 Operating1 行)。

Support load summary for Hanger at node 115B,

Load combination,Load (N),,

Sustained,-5188,,
Operating1,,,
Maximum,,,
Minimum,-5188,,

我正在使用 np.genfromtxt()。效果很好，除非我遇到列中的值少于 4 个或有时根本没有值的行。

我在 genfromtxt() 上使用 sum() - 请参阅代码。当只有一个值时，我使用了一个 float()。当没有时，我试图识别并将零分配给总数。我可以针对每种情况进行自定义，但想知道是否有一种通用的、更抽象的方法来读取和计算不可预测情况下的数字总和。

另外，我尝试了“missing_values”和“filling_values”，但它们似乎不起作用。如何计算文件中非零列的数量？

这是目前为止的部分代码:

def optimize(fn, optflag):

    modeltotals = []
    i=0
    csv1 = []
    j = 1 # line # count

    for line in csv.reader(filelist) :
    temp = repr(line)  

    if "Support load summary" in temp :
       csv1.append(line) # just making another list of actionable lines for future use
       if (d): print "\n", line
       continue
    if (optflag == "ope") : # optimize on Operating loads
       if "Operating1" in temp:
          csv1.append(line)
          if (len(line) > 4):
              modeltotals.append(sum(np.genfromtxt(line[1:], delimiter=",")))
              if (d): print "Sum of OPE Loads:", modeltotals[i], "\n"
          elif (len(line) > 0 and len(line) <= 4):
              if (d): print "line=", line, "length", len(line)
              line1 = np.genfromtxt(line[1:], delimiter=",")
              if not line1: # meaning if array is empty
                  modeltotals.append(0)
              else:
                  modeltotals.append(np.genfromtxt(line[1:], delimiter=",", missing_values=[0,0,0,0]))
              if (d): print "OPE Max:", modeltotals[i],"\n"

          i +=1
    elif (optflag == "minmax") : #optimize on all loads, min and max.
       #print  "i=", i
       if "Maximum" in temp:
          csv1.append(line)
          if (len(line) > 4):
              modeltotals.append(sum(np.genfromtxt(line[1:], delimiter=",")))
              if (d): print "Sum of Maxs:", modeltotals[i]
          elif (len(line) <= 4):
              #line1 = np.genfromtxt(line[1:], delimiter=",", filling_values = 0)
              #modeltotals.append(sum(line1))

              if (d): print "line=", line, "length", len(line)
              line1 = np.genfromtxt(line[1:], delimiter=",")
              print "line1 =", line1
              if not line1: # meaning if array is empty
                  modeltotals.append(0) 
              else:
                  modeltotals.append(np.genfromtxt(line[1:], delimiter=",", filling_values = 0))
              if (d): print "Max:", modeltotals[i]

          i+=1
       elif "Minimum" in temp:
          csv1.append(line)
          if (len(line) > 4):
              #print "#", j, "line", line
              modeltotals.append(sum(np.genfromtxt(line[1:], delimiter=",")))
              if (d): print "Sum of Mins:", modeltotals[i]

          elif (len(line) > 0 and len(line) <= 4):
              if (d): print "line=", line, "length", len(line)
              line1 = np.genfromtxt(line[1:], delimiter=",")
              if not line1: # meaning if array is empty
                  modeltotals.append(0)
              else:
                  modeltotals.append(np.genfromtxt(line[1:], delimiter=","))
              if (d): print "Min:", modeltotals[i]

          i +=1
    j+=1
 if len(modeltotals) > 0:
     print modeltotals
     average = float(sum(modeltotals))/len(modeltotals)  #sometimes error here
 else:
     return "000"  # error, seems like no file was analyzed
 if (d):
     print "Current model mean =", average

 del csv1[:]
 return abs(average)

我在不同的文件中遇到的几个错误都是类似的:

['Support load summary for restraint at node 20B', '']
Traceback (most recent call last):
File "sor4.py", line 190, in <module>
modelmean[filename] = optimize(filename, args.optimizeon)
  File "sor4.py", line 107, in optimize
   modeltotals.append(sum(np.genfromtxt(line[1:], delimiter=",")))
TypeError: iteration over a 0-d array

另一个错误是“无法转换为标量”。

我理解这些错误，但对 Python 的了解不多，无法巧妙地处理它们。很抱歉发了这么长的帖子；我会更好地更简洁地呈现信息。正如这里的另一位发帖人所说，我将很感激地接受你的回答。谢谢。

Answer 1

我将您的问题简化为以下代码。它检查 nans 和空输入字符串。

from StringIO import StringIO
import numpy as np

def getnumbers(s):
    try:
        res = np.genfromtxt(s, delimiter=",")
        return res[np.where(np.isnan(res), False, True)]
    except IOError as ioe:
        return np.array(0.)

print(sum(getnumbers(StringIO('1., 2., , '))))
print(sum(getnumbers(StringIO(''))))

给出结果

3.0
0.0