ruby - 巨大的 json 对象要出类拔萃？

Question

我有一个 PHP 应用程序，它有一个超过 10000 行的表，我试图通过我的 ROR 应用程序将其导出到 Excel 工作表中，但是我的请求在服务器上被 PHP 应用程序超时。所以我想知道有什么优雅的方法可以解决这个问题。我想出了两个解决方案。首先是进行批处理(需要阅读，因为我是新手)，另一个是我的 php 应用程序将发送一个大的 json 对象，我的 ruby​​ 应用程序将读取该 json 对象，将数据写入 excel 表并发回excel表。所以我想问问有没有更好的方法来处理这个问题？我如何将 json 转换为 excel 我在谷歌上搜索并发现 excel 转换为 json，但反之亦然。有什么建议吗？

Answer 1

我有一些时间，所以我在 ruby​​1.9 中构建了一个 json 到 excel csv 转换器:

它从一个文件中读取并将其写入另一个文件。

json2excel.rb

def json2excel(fromFile, toFile)
  pos = 0
  while true
    c = fromFile.read(1);pos += 1
    if c == ' ' or c == "\n" or c == "\r"
      # whitespace
    elsif c == '['
      # first bracket begins!
      attributes = []
      while true
        c = fromFile.read(1);pos += 1
        if c == '{'
          # now an object starts
          object = Hash.new
          while true
            puts "!!!"
            c = fromFile.read(1);pos += 1
            if c == '"'
              # new attribute starts
              name = ""
              while true
                c = fromFile.read(1);pos += 1
                if c == '"'
                  break
                else
                  name += c
                end
              end
              attributes << name unless attributes.include? name
              # scan for :
              while true
                c = fromFile.read(1);pos += 1
                if c == ':'
                  break
                elsif  c == ' ' or c == '\n' or c == '\r' # whitespace is ok
                else raise "4malformed json for excel conversion! char: #{c.inspect} position: #{pos}"
                end
              end
              # scan for staring value
              while true
                c = fromFile.read(1);pos += 1
                if c == '"'
                  # string follows
                  value = ""
                  value << c
                  while true
                    c = fromFile.read(1);pos += 1
                    value << c
                    if c == '"'
                      break
                    end
                  end
                  c = fromFile.read(1);pos += 1
                  break
                elsif  c == ' ' or c == '\n' or c == '\r' # whitespace is ok
                elsif "1234567890".include? c
                  # number follows
                  value = ""
                  value << c
                  while true
                    c = fromFile.read(1);pos += 1
                    if "1234567890".include? c
                      value << c
                    else break
                    end
                  end
                  break
                elsif c == "t"
                  # true follows
                  c = fromFile.read(3);pos += 3
                  if c != "rue"
                    raise "excpected true but found t#{c.inspect} position: #{pos}"
                  end
                  value = "true"
                  c = fromFile.read(1);pos += 1
                  break
                elsif c == "f"
                  # false follows
                  c = fromFile.read(4);pos += 4
                  if c != "alse"
                    raise "excpected false but found f#{c.inspect} position: #{pos}"
                  end
                  value = "false"
                  c = fromFile.read(1);pos += 1
                  break
                else raise "5malformed json for excel conversion! char: #{c.inspect} position: #{pos}"
                end
              end
              # value starts
              object[name] = value
              puts object
            end
            puts "c: #{c.inspect}"
            if c == "," 
              # comma is ok! just take many of them, does not hurt.
            elsif  c == ' ' or c == '\n' or c == '\r'
              # whitespace is ok
            elsif c == "}"
              # object ends!
              break
            else raise "3malformed json for excel conversion! char: #{c.inspect} position: #{pos}"
            end
          end
          attributes.each{|attr|
            value = object[attr]
            raise "expected object #{object} to have attribute #{attr} position: #{pos}" if value.nil?
            toFile.write(value)
            toFile.write(',')
          }
          toFile.write("\"\"\r\n") # this is the csv new line. a new object begins here
        elsif  c == ' ' or c == '\n' or c == '\r'
          # whitespace is ok
        elsif c == ']'
          attributes.each{ |attr|
            toFile.write(attr.inspect)
            toFile.write(",")
          }
          toFile.write("\"\"\r\n") # this is the csv new line. a new object begins here
          # the end of the file
          c = fromFile.read()
          if c != ''
            raise "end of listing was reached. skipping #{c.size} character after position #{pos}: #{c.inspect}"
          end
          break
        elsif c == ','
          # comma is ok! just take many of them, does not hurt.
        else
          raise "2malformed json for excel conversion! char: #{c.inspect} position: #{pos}"
        end
      end
      break
    else
      raise "1malformed json for excel conversion! char: #{c.inspect} position: #{pos}"
    end
  end
end

json2excel(File.open('json.txt'), File.open('excel.csv', 'wb'))

json.txt

[{"id": 1,"pro_id": 3,"pro_name": "asdf","cli_id": 113,"cli_name": "tyuryt"} , {"id": 1,"pro_id": 3,"pro_name": "asdf","cli_id": 113,"cli_name": "tyuryt"}]

excel.csv

1,3,"asdf",113,"tyuryt",""
1,3,"asdf",113,"tyuryt",""
"id","pro_id","pro_name","cli_id","cli_name",""

您的列名在文件末尾。

如果在第一个对象之后引入新属性，则并非所有列的元素计数都相等。

注意:它不会将所有内容加载到内存中，而是尽快将其写入文件。

它不做什么:

• 负数
• 带有 . 的数字
• 包含 " 的字符串。