python - 无法使用 xpath、lxml 从抓取的页面获取脚本标记的内容

Question

我正在编写一个 python 程序，作为其中的一部分，我需要从 google ngram 查看器中提取数据。例如搜索:

https://books.google.com/ngrams/graph?content=The+Godfather&year_start=1972-&year_end=2008&corpus=15&smoothing=3

我需要此标签的图表值:

  var data = [{"ngram": "The Godfather", "type": "NGRAM", "timeseries": [1.4183381225052472e-07, 1.4025288521679614e-07, 1.5749316872870622e-07, 1.618123600824869e-07, 1.7873649125834034e-07, 1.8325580697364785e-07, 1.838378673418057e-07, 1.7964884234191102e-07, 1.7921279850595185e-07, 1.8174738970953642e-07, 1.7919142944070439e-07, 1.7377866307859741e-07, 1.6968103417574249e-07, 1.6785447241675554e-07, 1.698323818085815e-07, 1.7567729011196726e-07, 1.7198440259237812e-07, 1.6793204338227952e-07, 1.7446684604952418e-07, 1.8290416343396438e-07, 1.8512590876136009e-07, 1.8604621183320497e-07, 1.9866517269357636e-07, 1.8994029866397405e-07, 2.0815326909736802e-07, 2.2254876138764042e-07, 2.2457939508058189e-07, 2.5743728875633156e-07, 2.7453169236326046e-07, 2.7866381507075335e-07, 3.0070588609630378e-07, 3.0204388273042629e-07, 2.9686912585345583e-07, 2.9399397760698776e-07, 2.7703356645740013e-07, 2.7225316614476467e-07, 2.6805425434872632e-07], "parent": ""}];
  if (data.length > 0) {
    ngrams.drawD3Chart(data, 1972, 2008, 1.0, "main");
  }

我尝试使用 lxml 将 HTML 转换为树，然后使用从网站复制的 xpath，如下所示:

page = requests.get('https://books.google.com/ngrams/graph?content=The+Godfather&year_start=1972-&year_end=2008&corpus=15&smoothing=3')
tree = html.fromstring(page.content)

nGramData = tree.xpath('//*[@id="container"]/script[11]/text()')

但是，如果我尝试打印 nGramData 我没有得到任何结果，我做错了什么或者有更好的方法吗？

Answer 1

使用以下工作代码:

>>> page = requests.get('https://books.google.com/ngrams/graph?content=The+Godfather&year_start=1972-&year_end=2008&corpus=15&smoothing=3')
>>> tree = html.fromstring(page.content)
>>> nGramData = tree.xpath('//*[@id="container"]/script')
>>> nGramData[6].text_content()
'\n  var data = [{"ngram": "The Godfather", "type": "NGRAM", "timeseries": [1.4183381225052472e-07, 1.4025288521679614e-07, 1.5749316872870622e-07, 1.618123600824869e-07, 1.7873649125834034e-07, 1.8325580697364785e-07, 1.838378673418057e-07, 1.7964884234191102e-07, 1.7921279850595185e-07, 1.8174738970953642e-07, 1.7919142944070439e-07, 1.7377866307859741e-07, 1.6968103417574249e-07, 1.6785447241675554e-07, 1.698323818085815e-07, 1.7567729011196726e-07, 1.7198440259237812e-07, 1.6793204338227952e-07, 1.7446684604952418e-07, 1.8290416343396438e-07, 1.8512590876136009e-07, 1.8604621183320497e-07, 1.9866517269357636e-07, 1.8994029866397405e-07, 2.0815326909736802e-07, 2.2254876138764042e-07, 2.2457939508058189e-07, 2.5743728875633156e-07, 2.7453169236326046e-07, 2.7866381507075335e-07, 3.0070588609630378e-07, 3.0204388273042629e-07, 2.9686912585345583e-07, 2.9399397760698776e-07, 2.7703356645740013e-07, 2.7225316614476467e-07, 2.6805425434872632e-07], "parent": ""}];\n  if (data.length > 0) {\n    ngrams.drawD3Chart(data, 1972, 2008, 1.0, "main");\n  }\n'