- android - 多次调用 OnPrimaryClipChangedListener
- android - 无法更新 RecyclerView 中的 TextView 字段
- android.database.CursorIndexOutOfBoundsException : Index 0 requested, 光标大小为 0
- android - 使用 AppCompat 时,我们是否需要明确指定其 UI 组件(Spinner、EditText)颜色
我正在尝试使用 Scrapy 从 Erowid.org(毒品旅行报告的精选数据库)中抓取数据。我正在使用 http://github.com/DavidYi1/Drug-Forum-Data-Mining 的修改版本。这使我能够抓取网站并为每种物质导出一个文件夹,其中每个行程报告都位于单独的 .txt 文件中。
我想获取有关药物剂量、摄入方法和体重的元数据,这些元数据通常在报告之前的表格中针对每次经历进行指示(例如 http://www.erowid.org/exp/69866 )。
但是,我不太确定如何执行此操作,以便可以在与每个报告对应的每个 .txt 文件的开头包含此元数据。例如:
substance: LSD
dose: 200ug
method: oral
weight: 160lb
Text of the trip report
提前致谢!
最佳答案
您可以使用 xpath 选择 dara:
In [49]: sub = response.xpath("//td[@class='dosechart-substance']/a/text()").extract_first()
In [50]: dose = response.xpath("//td[@class='dosechart-amount']/text()").extract_first()
In [51]: form = response.xpath("//td[@class='dosechart-form']/b/text()").extract_first()
In [52]: weight = response.xpath("//td[@class='bodyweight-amount']/text()").extract_first()
In [53]: sub, dose, form, weight
Out[53]: (u'LSD', u'6 hits', u'(blotter / tab)', u'170 lb')
文本位于 report-text-surround
类的 div 中:
In [64]: response.xpath("//div[@class='report-text-surround']/text()").extract()
Out[64]:
[u'\n',
u'\n\n',
u'\n\t',
u'\n',
u'\n\n\t',
u'\n\n',
u'\n',
u'\n',
u'\nI became interested in psychedelic drugs in the winter of 2005, and having had multiple experiences with mushrooms and LSD at lower doses, I decided during mid March of 2006 that I was ready for a step up. I spent a few hours on Saturday morning cleaning up, I ate a light meal (I believe it was cold pizza) and at 2:40 in the afternoon I ate my last 6 hits of blotter acid. \r',
u'\n\r',
u'\nI considered myself a fairly seasoned psychonaut at the time, and I recognized the initial onset of the condition roughly fifteen minutes later. Colors became sharper, and sounds took on a strange ethereal quality. I walked into the den, encouraged. It was a large room, with a hardwood floor and a fireplace. I felt certain that this was the right room in which to begin my journey, and so I reclined on the couch and fell into a relaxed, mellow state. That was when I began to hear singing. It started as a sweet humming sound, like a barbershop quartet tuning up, only at very low volume. The sound began to rise, and the tone began to change. The windows were slightly open, and a cool breeze began blowing the curtains from the window, It seemed to me at the time that there was a whole invisible choir in the room, and each change of the light wrought its own change upon the tune. It grew and grew, and soon instruments were thrown into this beautiful choral mix. I began to search in vain for the source of the sound, but as I moved the music grew around me. Soon I recognized that the sound was coming from me, and from all the life around me. The music was the embodiment of life, and spirit, and joy. Everything I saw added to this beautiful melody. I went into another room, a hallway this time, and a few soaring violin notes accompanied me. The mellow feeling gave way to a kind of curious energy, as if I was using awe itself as fuel. \r',
u'\n\r',
u"\nFor the next thirteen hours, I was enthralled by the beauty of life as I had not been in over a decade. Color seemed to thrive in this new, alien world! The entirety of the rainbow glared at me from every corner. I saw colors I couldn't name, colors that I've never seen replicated. It was as if I had never seen color. I looked out the window towards the bay, and grasped the size of the universe in a new way. A vast map formed inside my head, earth, and the planets, and the sun and stars and space beyond. It grew ever larger, and an arrow appeared over the grain of sand the earth had become, an arrow with the words 'You are Here' hovering almost comically above. I snapped back to reality, or what my reality had become, and took a few moments to clear my head. While I was recovering from the sheer size of the void, a white swing caught my eye. A red cardinal had perched on the right hand arm-rest and the colors were so sharp and clean that they almost hurt to look at. I thought it seemed very realistic, then laughed to myself. After all, it was real life! My mental processes seemed remarkably clear, and I began to 'toy' with ideas in my head. I was fascinated by my new found ability to take a blueprint, and picture it as an actual object. I am not nor will I ever be a mechanically- inclined person, but today it seemed not to matter. I built entire houses out of thought, being able to cross-section and dissect them with ease. Even such complex instruments as jet engines could be effortlessly mentally re-constructed. After playing in the sandbox that my brain had become for some time, I opened the back door and headed out onto the patio.\r",
u'\n\r',
u"\nTrees and grass were moving in a way I had never seen before, above the surface of the ocean. Branches and leaves seemed to burn with motion, each becoming a hazy, insubstantial blur as they moved. 'Like kelp, but not' I said out loud. It occurred to me that I was talking to myself, and had been for quite some time, and a fresh gale of laughter overtook me. Each old and familiar sight was altered, changed somehow as to make it appear completely new and wonderful. I looked up at the sky, curious, and was greeted by the sight of a vast cerulean ocean. It was a very clear day, and both the moon and the sun were visible, and they both screamed and competed for my attention. A jet soared overhead, and it left a great plume of rainbow exhaust all the way across the sky. As I watched, the jet passed cleanly through the moon and out the other side, creating momentary chaos. The sky changed entirely, becoming first darker and then lighter. Colors sprang from nowhere, and disappeared just as quickly. I knew how easy it would be to forget everything and watch the sky patiently until the end of time, but I had other things planned for the day. I looked at an oak, and saw that each leaf was in fact a diffuse, green light. The green gave way to blue, which in turn gave way to yellow. These changes continued at a fairly predictable rate, and I realized that I had control over them! \r",
u'\n\r',
u"\nSuddenly it seemed that the outside world was on my side, and I was safe and in control. I felt completely at peace. It was a warm day, but even if it had been freezing I doubt if I would have felt it. Eventually I decided to return to the house before I forgot how to move. It sounds silly to me looking back now, but at the time it seemed to be very possible. Given the sensory over stimulation of the situation, I'm not entirely surprised. \r",
u'\n\r',
u'\nI ventured back inside and turned the t.v on, browsing at large. It started to become complicated as the buttons began to change, sometimes as part of a pattern, sometimes at random. A short time later, they began to drift from the face of the remote itself! Laughing like a loon, I tossed the oddity of a remote back onto the table, where the clean grain of the cherry wood swam over it and devoured it. I smirked at this odd sight, completely at ease, and refocused my attention on the movie. It was an old favorite of mine, but I found vision being routinely drawn away to a large, white wall on my right. I eventually gave up watching the t.v to focus my full attention on the wall. Complicated fractal and runic patterns would arise spontaneously on its surface, only to be seconds later subducted into the ether. Chaos would destroy order, which would instantly arise in another form. I saw planets rise and fall, stars form, reason melt and the very fabric of the universe be torn open, to spontaneously be pulled back into the plasma of the wall, and then spit back out in another form. At times I could only interpret the colors as noise, the sounds as color. I could visibly see the music from the clock radio on the counter traveling from point to destination, but I could not for the life of me make out the digital readout of the clock. I became very preoccupied with time, and dimension. Such intangible concepts to me were like solid objects, visible things that could be touched and seen. \r',
u'\n\r',
u"\nIf it were possible to break an acid trip down into words, the sheer volume of words would be staggering. Hundreds of ideas cross the brain in seconds, weaving a complicated spider's web that seem to fit perfectly together, although they share few common characteristics. Color and light seem as different sides of the same coin, striving in some cases for balance, and in other cases for contrast. I decided to listen to some music without the radio, and in seconds was conducting full symphonies in my head, adding and changing instruments on the fly to produce intricate and pleasing melodies. All the electronic gadgets in the world seemed inferior to that wonderful internal toolkit I was only now receiving full access to. Soon the audible mental orchestra gave way to a visible one, and I imagined myself to be in a concert hall, watching a group of celestial beings play instruments unlike any I had ever seen before. It was purely a mental picture, but something in its aspect reminded me so strongly of real life that I began to wonder if I had been granted a true vision of some other dimension. Realizing the answer to this question would always escape me, I endeavored to think it over anyway, with disappointing results After some time of this, I realized it was quickly approaching sunset, a sight I did not intend to miss. \r",
u'\n\r',
u"\nI ventured outside again and felt my breath leave my throat. I was enraptured. The sky that had been so perfectly, absolutely blue only a few hours ago was now a violet so gorgeous and benign that the strength left my knees and I sat loudly on the ground with a loud 'oomph!' sound. This violet color was reflected on the surface of the water as an intense, fiery orange. The sun itself had turned a red so deep it seemed a burning ball of blood in the sky. All around, waves of colored smoke-light (it had the characteristics of both) began to caper and dance. They distinctly seemed to be sentient, and benevolent. I reached my hand into one of them and at that point I had an epiphany. I realized that nothing is unimportant, and everything matters. For some people this might seem obvious, but the idea struck me at the time as being crucial, and central to my own personal beliefs. The sun finished its performance, and the stars made their way onto the stage. The sky became a vast, black painted sheet of canvas, and the stars gleamed like cold jewels in the night sky. They rippled and undulated, as if they were pleased with their own beauty. \r",
u'\n\r',
u"\nI sat there, totally unable to move for an indeterminate amount of time, watching the night silently explode with color and life. Beauty enveloped me, rolled over me like a wave. The clouds began to race across the shining orange surface of the moon, surrounding it in a halo of the purest gold. This gold light bounced off the surface of the water, lending an silver gleam to every moist blade of grass. The night itself seemed to beckon me, it was totally engrossing. I watched and I walked, but I didn't speak, not wanting the spoken word to mar the beauty of the night world. I wandered for a while, oblivious to both the growing coolness in the air and the emptiness of my stomach. I didn't seem to need food, the night provided more than enough to sustain me. I wandered for hours, thinking of many things, and basking in the beauty of the night. \r",
u'\n\r',
u'\nEventually, the maniac energy which had possessed me throughout the day gave way to an exalted, happy weariness, much like the one that accompanies a a day of hard but ultimately successful labor. I retired to my bedroom, and began to stare at the ceiling. Chunks of drywall began to bulge and drip from the walls and ceiling, only to dissipate into a thin smoke. The smoke would drift a few feet from its point of origin, and then lazily make its way back into the gap. Soon enough, I began to ponder the nature of the universe in a general way, which led me specifically to thoughts of black holes. Not a moment after I had begun this train of thought, a small black vortex opened up in the center of the ceiling with an audible stretching noise. It grew larger and larger, and upon reaching the size of a basketball began to exude ribbons of the colored smoke-light that I had seen earlier. They were very bright, primary colors, the exact shades on a kindergarten color chart. They began to revolve around the room, picking up speed as they went. Eventually the colors began to band together, and soon the room was a circus of colors all racing through each other and intertwining. All at once, the mystery of the tie-dyed t shirt was no mystery to me. The night ended with me laughing myself to sleep. \r',
u'\n\r',
u"\nI awoke the next morning with a sense of exultation, I felt like I'd lived eons and seen things that no one else would ever see. The relief of stress was enormous, and for a few months the minor problems we all have every day failed to dampen my new found spirits. It has affected me in more ways that I can put into words, but overall I'm a much more tolerant person, and I like to think of myself as being more understanding. I feel as if the mystery has returned to my life, and with it my will to continue moving forward. After all, there's no telling what might be around the next bend, and whether or not it'll be able to top that day as the most thrilling and awe-inspiring of my life.",
u'\n',
u'\n\n\n\n',
u'\n\n']
如果您查看源代码,您可以看到数据所在的标签,您只需根据该标签进行选择即可:
您可能还想阅读 xpath tutorial
关于python - 使用 scrapy 从药物报告表中抓取数据,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/37284682/
@After public void afterScenario() { if (ScenarioManager.getScenario().isFailed()) {
我已将 BIRT 报告集成到 Grails 中并设计了一份报告。我的 grails 应用程序中有一个名为 startPeriod (仅限月份和年份)的参数,我想将其传递给 BIRT。然后 BIRT 调
我有一些 Oracle 报告 (.rdf),正在考虑将其转换为 BIRT 报告。有没有办法将 .rdf 文件转换为 BIRT 报告设计文件? 最佳答案 完全自动化的解决方案可能是不可能的。您可以部分自
当 gcc 4.1(使用 gcov)下一行: p = 新类; 报告为 100% 分支覆盖率 为什么? 因为启用了异常处理!!! 为了解决此问题,请指定: -fno-exceptions 在 g++
真的有好 免费 BugZilla 报告工具?我发现 Web 界面上的默认搜索选项太有限了。我最大的问题是缺少 Order By 选项(一次只有 1 个字段,可供选择的字段集非常有限)。我已经做了一些谷
是否可以在 CFMX7 上运行 ColdFusion Report builder 生成的报告? 更明确地说,是否可以将 CF7 中的报告生成引擎更改为 CF8? 最佳答案 我猜这可能很难做到。我记得
根据Lucintel发布的新市场报告,智能家居市场的未来看起来很有吸引力,在家用安全、家电、娱乐、照明、HVAC、医疗保健和厨房应用中将带来许多机遇。 由于COVID-19导致的全球经济衰退,
PHPCodeSniffer 是否生成 HTML 报告? 如果不是呢?怎么办? 目前,我可以运行 PHPCodeSniffer,但它只生成 XML 文件并在终端中显示结果。 如何在 phpunit 中
我在一个包中添加了一个简单的测试。 按照手册中的建议,我尝试让 PHPUnit 加载配置: phpunit -c /app phpunit.xml 看起来像这样:
我有两个从 csv 文件加载的数据框。基本上来自不同的环境但格式/列相似,它们的行/值可能有所不同。我想找到差异并在新的数据框中创建它们。两个数据框也将具有相同的顺序。我有 100 个要比较的文件。提
我想看看是否有办法通过 javadoc 在我的 junit 报告中包含“描述性文本”。 JUnit 4 似乎不像 TestNG 那样支持 @Test 注释的“描述”属性。 到目前为止,我所研究的只有一
我正在使用操作、 Controller 、servlet struts 框架编写 Excel 报告。该报告非常拥挤,已经有大约 10 个单独的查询。由于报告发生变化,我需要再添加大约 10 个查询。有
在放弃 Syleam 的 openerp jasper 模块后,我在 Nan Tic 的 jasper_reports 模块上苦苦挣扎。 它一直给我一个错误: File "C:\Program Fil
我希望创建一个简单的日历。每天由编码器生成条目计数并以日历样式查看。如一月、二月等。或按月显示全年。 database have date_added and encoder columns 我在将它
我必须为报告创建 MySQL 查询。 我有一个表history,它记录产品订单的状态更改。我有订单生命周期(订单流程)的以下状态:新、已确认、正在处理、已发货、已交付、已取消、已退回。订单不一定遵循此
如何将多个查询合并为一个? 例如: //Successful Sales: SELECT username, count(*) as TotalSales, sum(point) as Points
MySQL 优化技术的新手。请找到下面的 mysqltuner.pl 报告,并建议我应该更改 my.cnf 中的哪些变量以优化性能。 还有一个问题- 我无法在我的 my.cnf 中找到一些变量,例如
我想知道,我想将我的 Swing Worker 的某种形式的进度报告回主线程,以便我的界面可以使用随着进度增加而变化的标签进行更新,例如 checking 1/6... checking 2/6...
我正在尝试在“报告”>“销售”下运行 Magento Paypal 结算报告,但每次我尝试运行该报告时,我都会收到消息“由于配置为空,无法获取任何内容” 我查看了“系统”>“配置”>“销售”>“付款方
我想要一个工具来帮助创建 sql 查询(对于非 IT 人员),例如 dbforge。 我希望我们的非 IT 人员(例如运营)创建他们自己的 sql 查询。 我的第二个目标是让他们能够按需执行这些查询。
我是一名优秀的程序员,十分优秀!