- html - 出于某种原因,IE8 对我的 Sass 文件中继承的 html5 CSS 不友好?
- JMeter 在响应断言中使用 span 标签的问题
- html - 在 :hover and :active? 上具有不同效果的 CSS 动画
- html - 相对于居中的 html 内容固定的 CSS 重复背景?
获取包含特定字符串的链接:
这是我的程序:
package com.dcvsolution.crawler.songkhoedotvn;
import java.io.IOException;
import java.util.HashSet;
import java.util.Set;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import com.dcvsolution.commons.CommonUtil;
public class GetCategories {
public static void main(String[] args) throws IOException {
// Bait link.
String baitLink = "http://songkhoe.vn/";
// If we prefer mobile-site option (default of "songkhoe.vn" when use
// Jsoup).
// Document doc = Jsoup.connect(url).get();
// If we prefer full-page format web-page option.
// Đánh lừa trang web, đây là đang duyệt web trên PC, là Google.
Document doc = Jsoup.connect(baitLink)
.userAgent("Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0")
.referrer("http://www.google.com").get();
Elements links = doc.select("a[href]");
// Push data to a set for unique result.
Set<Element> uniqueSet = new HashSet<Element>(links);
// Iterate through above set.
for (Element link : uniqueSet) {
CommonUtil.printByFoundry("%s", link.attr("abs:href"), CommonUtil.trim(link.text(), 3500));
// Posts post = new Posts();
// post.setTitle(link.attr("abs:href"));
// post.setContent(link.attr("abs:href"));
// post.setCreateTimeDb(new Date());
// Session session =
// HibernateUtil.getSessionFactory().openSession();
// session.beginTransaction();
// session.save(post);
// session.getTransaction().commit();
}
}
}
我有一个列表:
http://songkhoe.vn/chuyen-muc-vui-khoe-s2953-0.html
http://songkhoe.vn/su-nguy-hiem-tu-nhung-ke-mac-benh-au-dam-benh-hoan-s2959-1192-311862.html
http://songkhoe.vn/chuyen-trang-loi-khuyen-chuyen-gia-s21253-0.html
http://songkhoe.vn/chi-tiet-10-thuc-pham-chua-viem-hong-hieu-qua-khong-ngo-s2529-604-482929.html
http://tinngan.vn
http://songkhoe.vn/dien-dan-benh-dau-s25.html
http://songkhoe.vn/dien-dan-di-vat-trong-tai-s2529-609.html
http://songkhoe.vn/hang-tram-bac-si-xuong-duong-van-dong-hien-mau-tinh-nguyen-s21237-0-297990.html
http://songkhoe.vn/chi-tiet-cach-phong-ngua-benh-mach-vanh-s2480-502-483072.html
http://songkhoe.vn/dien-dan-ung-thu-tuyen-tuy-s21008-860.html
http://viettelstudy.vn
http://songkhoe.vn/home.html
http://songkhoe.vn/tu-van-truc-tiep-lam-gi-de-ung-pho-voi-vi-rut-zika-s21256-0-293851.html
http://songkhoe.vn/dau-hieu-benh-nguy-hiem-tu-mong-tay-s2964-0-311784.html
http://songkhoe.vn/bac-si-cong-tac-did33.html
http://songkhoe.vn/rssMain.html
http://songkhoe.vn/tom-cruise-tam-guong-sang-ve-no-luc-vuot-qua-benh-tat-s2953-0-311765.html
http://songkhoe.vn/neu-ban-lo-lang-gio-cha-co-han-the-de-doa-suc-khoe-gia-dinh-s2955-1178-311831.html
http://songkhoe.vn/chuyen-muc-video-meo-hay-s25-0.html
http://songkhoe.vn/chuyen-muc-video-bai-tap-van-dong-s21-0.html
http://songkhoe.vn/chi-tiet-phat-hien-som-dau-hieu-cua-thai-ngoai-tu-cung-s2750-807-482288.html
http://songkhoe.vn/lien-he-hop-tac.html
http://songkhoe.vn/chuyen-muc-tu-van-truc-tiep-tuan-s21256-0-1.html
http://songkhoe.vn/bac-si-tu-van-lam-the-nao-de-be-ngu-het-dem-sau-va-ngon-giac-s2539-1131-311461.html
http://songkhoe.vn/ths-dinh-van-tai-dia-diem-cu-tru-mun-herpes-sinh-duc-s21253-0-309620.html
http://songkhoe.vn/bac-si-cong-tac-did40.html
http://songkhoe.vn/dien-dan-hoi-dap.html
http://songkhoe.vn/bac-si-tu-van-dau-hieu-phan-biet-tre-tu-ky-va-cham-phat-trien-s2962-697-311489.html
http://songkhoe.vn/cho-tre-choi-nha-phao-vui-nhung-nguy-hiem-rinh-rap-s2960-1185-311760.html
http://songkhoe.vn/bac-si-cong-tac-did25.html
http://songkhoe.vn/6-sai-lam-khien-benh-viem-xoang-viem-mui-ngay-cang-nang-hon-s2964-0-267073.html
http://songkhoe.vn/song-lau-song-khoe-nho-5-thoi-quen-don-gian-s21237-0-301435.html
http://songkhoe.vn/chu-de-y-te-noi-bat-dau-mat-do-s215.html
http://songkhoe.vn/bac-si-cong-tac.html
http://songkhoe.vn/dien-dan-chan-thuong-khi-quan-s2533-1304.html
http://songkhoe.vn/chu-de-y-te-noi-bat-benh-do-vi-rut-zika-s247.html
http://songkhoe.vn/tu-van-can-thiep-tim-mach-giup-hoi-phuc-nhanh-it-bien-chung-s21256-0-309613.html
http://songkhoe.vn/bac-si-tu-van-can-65-di-op-co-deo-kinh-ap-trong-duoc-khong-s2530-623-311442.html
http://songkhoe.vn/dien-dan-chan-thuong-thanh-quan-s2529-1305.html
http://songkhoe.vn/cac-bai-tap-don-gian-giup-nu-gioi-so-huu-lung-thon-goi-cam-s2953-0-311797.html
http://songkhoe.vn/bs-nguyen-vu-cam-tu-con-duong-lay-nhiem-benh-dau-mat-do-s2964-0-189600.html
http://songkhoe.vn/bac-si-tu-van-sau-khi-tiem-phong-vac-xin-5in1-quinvaxem-be-bi-soi-bung-di-phan-long-s2534-566-311465.html
http://songkhoe.vn/video-tac-hai-khi-vo-tu-chia-se-thong-tin-ca-nhan-tren-mang-xa-hoi-s225-5014.html
http://songkhoe.vn/chuyen-muc-lam-me-s2960-0.html
http://songkhoe.vn/bac-si-cong-tac-did40.html
http://songkhoe.vn/tu-van-can-thiep-tim-mach-giup-hoi-phuc-nhanh-it-bien-chung-s21256-0-309613.html
http://songkhoe.vn/16-ly-do-khong-ngo-khien-ban-stress-s2953-1174-137869.html
http://songkhoe.vn/chu-de-y-te-noi-bat-dot-quy-s231.html
http://songkhoe.vn/bac-si-tu-van-dau-hieu-phan-biet-tre-tu-ky-va-cham-phat-trien-s2962-697-311489.html
http://songkhoe.vn/tra-cuu-nha-thuoc.html
http://songkhoe.vn/kinh-nghiem-cham-chau-tu-ky-ngu-ngon-cua-ba-noi-s2960-1185-311815.html
http://songkhoe.vn/bac-si-cong-tac-did57.html
http://songkhoe.vn/chuyen-trang-song-khoe-manh-s21254-0.html
http://songkhoe.vn/bac-si-cong-tac-did49.html
http://songkhoe.vn/bs-le-huy-tuan-pha-thai-noi-khoa-thuong-ra-mau-tu-10-den-15-ngay-s21253-0-310678.html
http://songkhoe.vn/uong-nuoc-dung-gio-khoe-manh-khong-ngo-s2955-1176-311770.html
http://songkhoe.vn/chuyen-muc-dinh-duong-s2955-0.html
http://songkhoe.vn/chuyen-muc-video-y-hoc-thuong-thuc-s22-0.html
http://songkhoe.vn/kich-thuoc-duong-vat-anh-huong-the-nao-den-tinh-duc-va-sinh-san-s2959-1192-311791.html
http://songkhoe.vn/hay-dung-lai-ngay-viec-dan-mat-vao-dien-thoai-boi-le-s2953-0-311820.html
http://songkhoe.vn/bac-si-cong-tac-did25.html
http://songkhoe.vn/dien-dan-chan-thuong-so-nao-s2536-701.html
http://songkhoe.vn/video-canh-bao-tre-co-the-bi-bat-coc-chi-boi-1-cau-noi-cua-nguoi-la-s225-5009.html
http://songkhoe.vn/chi-tiet-dau-hieu-ho-canh-bao-nhieu-benh-duong-ho-hap-s2529-746-485337.html
http://songkhoe.vn/video-canh-bao-tre-co-the-bi-bat-coc-chi-boi-1-cau-noi-cua-nguoi-la-s225-5009.html
http://songkhoe.vn/giat-quan-ao-cho-tre-so-sinh-dung-tuong-de-s2960-1185-311866.html
http://songkhoe.vn/phu-nu-can-trong-voi-tac-voi-trung-thu-pham-gay-vo-sinh-s2959-1193-311825.html
http://songkhoe.vn/bac-si-cong-tac-did53.html
http://songkhoe.vn/trang-dem-dieu-che-hon-9000-don-vi-mau-s21237-0-298643.html
http://songkhoe.vn/gui-cau-hoi.html
http://songkhoe.vn/video-6-dieu-can-phai-biet-ve-vi-rut-zika-s223-5020.html
http://songkhoe.vn/video-ung-thu-dai-truc-trang-tat-tan-tat-nhung-dieu-can-phai-biet-s223-5016.html
http://songkhoe.vn/bac-si-cong-tac-did70.html
http://tiin.vn
http://songkhoe.vn/cong-van-tang-cuong-phong-chong-viem-nao-nhat-ban-s2964-0-171284.html
http://songkhoe.vn/dien-dan-viem-ruot-thua-cap-s2534-1083.html
http://songkhoe.vn/hoang-mang-gao-nau-com-de-qua-dem-chuyen-mau-do-quach-s2961-1186-311822.html
http://songkhoe.vn/su-nguy-hiem-tu-nhung-ke-mac-benh-au-dam-benh-hoan-s2959-1192-311862.html
http://songkhoe.vn/hay-dung-lai-ngay-viec-dan-mat-vao-dien-thoai-boi-le-s2953-0-311820.html
http://songkhoe.vn/lam-dong-canh-bao-muc-do-2-dich-benh-do-vi-rut-zika-s21254-0-311804.html
http://songkhoe.vn/chuyen-muc-video.html
http://songkhoe.vn/chuyen-trang-loi-khuyen-tu-chuyen-gia-s21253-0.html
http://songkhoe.vn/xem-theo-doi-tuong.html
http://songkhoe.vn/video-ung-thu-dai-truc-trang-tat-tan-tat-nhung-dieu-can-phai-biet-s223-5016.html
http://songkhoe.vn/dien-dan-teo-thuc-quan-s2534-1313.html
http://songkhoe.vn/hang-tram-bac-si-xuong-duong-van-dong-hien-mau-tinh-nguyen-s21237-0-297990.html
https://www.facebook.com/songkhoe.com.vn
http://songkhoe.vn/bs-le-huy-tuan-pha-thai-noi-khoa-thuong-ra-mau-tu-10-den-15-ngay-s21253-0-310678.html
http://songkhoe.vn/bac-si-cong-tac-did35.html
http://songkhoe.vn/doc-tin.html
http://songkhoe.vn/kho-tieu-nhung-bieu-hien-rat-de-nhan-ra-s2964-0-311790.html
http://songkhoe.vn/chuyen-muc-gioi-tinh-s2959-0.html
http://songkhoe.vn/bac-si-tu-van-bi-xuat-huyet-nao-nhung-khong-mo-duoc-vi-suc-khoe-yeu-co-nguy-hiem-khong-s2536-701-311495.html
http://songkhoe.vn/tu-van-truc-tuyen-ap-dung-tien-bo-cua-y-hoc-trong-phau-thuat-tim-bam-sinh-s21256-0-281134.html
http://songkhoe.vn/dien-dan-hoi-dap.html
http://songkhoe.vn/chuyen-muc-gioi-tinh-s2959-0.html
http://songkhoe.vn/hoang-mang-gao-nau-com-de-qua-dem-chuyen-mau-do-quach-s2961-1186-311822.html
http://songkhoe.vn/dien-dan-trat-khop-vai-s2528-1291.html
http://songkhoe.vn/cac-bai-tap-don-gian-giup-nu-gioi-so-huu-lung-thon-goi-cam-s2953-0-311797.html
http://songkhoe.vn/xem-theo-van.html
http://keeng.vn
http://songkhoe.vn/chuyen-muc-video.html
http://songkhoe.vn/bac-si-tu-van-sau-khi-tiem-phong-vac-xin-5in1-quinvaxem-be-bi-soi-bung-di-phan-long-s2534-566-311465.html
http://songkhoe.vn/chu-de-y-te-noi-bat-stress-s221.html
http://songkhoe.vn/cho-tre-choi-nha-phao-vui-nhung-nguy-hiem-rinh-rap-s2960-1185-311760.html
http://songkhoe.vn/chuyen-muc-can-biet-s2964-0.html
http://songkhoe.vn/tra-cuu-phong-kham.html
http://songkhoe.vn/bs-le-huy-tuan-dau-hieu-bi-nhiem-nam-candida-s21253-0-309618.html
http://songkhoe.vn/video-tac-hai-khi-vo-tu-chia-se-thong-tin-ca-nhan-tren-mang-xa-hoi-s225-5014.html
http://songkhoe.vn/chu-de-y-te-noi-bat.html
http://songkhoe.vn/bac-si-cong-tac-did15.html
http://songkhoe.vn/giat-quan-ao-cho-tre-so-sinh-dung-tuong-de-s2960-1185-311866.html
http://songkhoe.vn/kho-tieu-nhung-bieu-hien-rat-de-nhan-ra-s2964-0-311790.html
http://songkhoe.vn/bac-si-tu-van-phau-thuat-tham-my-co-the-het-lac-va-nhin-binh-thuong-khong-s2530-629-311452.html
http://songkhoe.vn/chuyen-muc-lam-me-s2960-0.html
http://songkhoe.vn/700-benh.html
http://songkhoe.vn/bac-si-cong-tac-did18.html
http://songkhoe.vn/lam-dong-canh-bao-muc-do-2-dich-benh-do-vi-rut-zika-s21254-0-311804.html
http://songkhoe.vn/kich-thuoc-duong-vat-anh-huong-the-nao-den-tinh-duc-va-sinh-san-s2959-1192-311791.html
http://songkhoe.vn/700-benh.html
http://songkhoe.vn/chuyen-trang-song-khoe-manh-s21254-0.html
http://songkhoe.vn/giat-quan-ao-cho-tre-so-sinh-dung-tuong-de-s2960-1185-311866.html
http://songkhoe.vn/chuyen-muc-can-biet-s2964-0.html
http://songkhoe.vn/nguoi-nghi-sot-xuat-huyet-nhe-can-xet-nghiem-tam-soat-zika-s2961-1187-311056.html
http://songkhoe.vn/chuyen-muc-tu-van-truc-tiep-tuan-s21256-0-1.html
http://songkhoe.vn/bac-si-tu-van-can-65-di-op-co-deo-kinh-ap-trong-duoc-khong-s2530-623-311442.html
http://songkhoe.vn/xem-theo-chuyen-khoa.html
http://songkhoe.vn/dien-dan-viem-hong-do-lien-cau-s2529-1231.html
http://songkhoe.vn/video-6-dieu-can-phai-biet-ve-vi-rut-zika-s223-5020.html
http://songkhoe.vn/bac-si-cong-tac-did57.html
http://songkhoe.vn/bac-si-cong-tac-did70.html
http://songkhoe.vn/chu-de-y-te-noi-bat-viem-xoang-viem-mui-di-ung-s212.html
http://songkhoe.vn/dien-dan-co-that-thuc-quan-s2534-921.html
http://songkhoe.vn/cuba-xac-nhan-truong-hop-mang-thai-dau-tien-nhiem-zika-s2961-1187-311869.html
http://songkhoe.vn/bac-si-tu-van-bi-xuat-huyet-nao-nhung-khong-mo-duoc-vi-suc-khoe-yeu-co-nguy-hiem-khong-s2536-701-311495.html
http://songkhoe.vn/home.html
http://songkhoe.vn/kho-tieu-nhung-bieu-hien-rat-de-nhan-ra-s2964-0-311790.html
http://songkhoe.vn/nguy-co-dot-quy-nao-do-gio-lam-viec-keo-dai-s2964-0-266576.html
http://songkhoe.vn/hit-tho-de-tieu-mo-thua-s21254-0-311233.html
http://songkhoe.vn/gioi-thieu.html
http://songkhoe.vn/bac-si-cong-tac-did33.html
http://songkhoe.vn/truc-tiep-tu-van-xu-ly-cac-ca-kho-trong-dieu-tri-tim-bam-sinh-s21256-0-296488.html
http://songkhoe.vn/chuyen-muc-tam-hong-s21237-0.html
http://songkhoe.vn/dien-dan-thoat-vi-ron-s2534-1077.html
http://songkhoe.vn/tap-gym-kieu-nay-chang-co-loi-gi-ma-con-hong-dang-s21254-0-311149.html
http://songkhoe.vn/gui-cau-hoi.html
http://songkhoe.vn/phu-nu-can-trong-voi-tac-voi-trung-thu-pham-gay-vo-sinh-s2959-1193-311825.html
http://songkhoe.vn/bac-si-cong-tac-did35.html
http://songkhoe.vn/sitemap.html
http://songkhoe.vn/chu-de-y-te-noi-bat-viem-nao-nhat-ban-s234.html
http://songkhoe.vn/bac-si-cong-tac-did15.html
http://songkhoe.vn/phu-nu-can-trong-voi-tac-voi-trung-thu-pham-gay-vo-sinh-s2959-1193-311825.html
http://songkhoe.vn/bac-si-tu-van-vo-mang-bau-tuan-22-xi-mui-ra-1-cuc-mau-dong-co-nguy-hiem-khong-s2529-613-311462.html
http://songkhoe.vn/chuyen-muc-vui-khoe-s2953-0.html
http://songkhoe.vn/video-tu-van-can-thiep-tim-mach-giup-hoi-phuc-nhanh-it-bien-chung-p1-s231-5017.html
http://songkhoe.vn/bac-si-tu-van-vo-mang-bau-tuan-22-xi-mui-ra-1-cuc-mau-dong-co-nguy-hiem-khong-s2529-613-311462.html
http://songkhoe.vn/bac-si-tu-van-phau-thuat-tham-my-co-the-het-lac-va-nhin-binh-thuong-khong-s2530-629-311452.html
http://songkhoe.vn/su-nguy-hiem-tu-nhung-ke-mac-benh-au-dam-benh-hoan-s2959-1192-311862.html
http://songkhoe.vn/home.html
http://songkhoe.vn/uong-nuoc-dung-gio-khoe-manh-khong-ngo-s2955-1176-311770.html
http://songkhoe.vn/chuyen-muc-tam-hong-s21237-0.html
http://songkhoe.vn/700-benh.html
http://songkhoe.vn/cuba-xac-nhan-truong-hop-mang-thai-dau-tien-nhiem-zika-s2961-1187-311869.html
http://songkhoe.vn/chuyen-muc-dinh-duong-s2955-0.html
http://songkhoe.vn/chuyen-muc-video-cuoi-s27-0.html
http://songkhoe.vn/chuyen-muc-video-tu-van-truc-tuyen-s232-0.html
http://songkhoe.vn/video-tu-van-can-thiep-tim-mach-giup-hoi-phuc-nhanh-it-bien-chung-p1-s231-5017.html
http://songkhoe.vn/bs-nguyen-vu-cam-tu-con-duong-lay-nhiem-benh-dau-mat-do-s2964-0-189600.html
http://songkhoe.vn/dien-dan-viem-loet-mieng-s2538-756.html
http://songkhoe.vn/tra-cuu-benh-vien.html
http://songkhoe.vn/nguyen-gd-bv-k-tw-chi-dan-cach-tu-kham-de-phat-hien-ung-thu-s2964-0-311810.html
http://songkhoe.vn/6-sai-lam-khien-benh-viem-xoang-viem-mui-ngay-cang-nang-hon-s2964-0-267073.html
http://songkhoe.vn/cong-van-tang-cuong-phong-chong-viem-nao-nhat-ban-s2964-0-171284.html
http://songkhoe.vn/tra-cuu-benh-vien.html
http://songkhoe.vn/16-ly-do-khong-ngo-khien-ban-stress-s2953-1174-137869.html
http://songkhoe.vn/10-nguoi-bi-ung-thu-4-nguoi-vi-su-dung-thuc-pham-ban-s2955-0-311795.html
http://songkhoe.vn/bac-si-cong-tac-did18.html
http://songkhoe.vn/chuyen-de-kinh-ngac-nhung-dieu-ky-la-ve-sex-s2211.html
http://songkhoe.vn/nguoi-nghi-sot-xuat-huyet-nhe-can-xet-nghiem-tam-soat-zika-s2961-1187-311056.html
http://songkhoe.vn/dien-dan-xuong-thuy-tinh-s2528-591.html
http://songkhoe.vn/chuyen-muc-video-bai-tap-yoga-hang-ngay-s26-0.html
http://songkhoe.vn/dien-dan-xuat-tinh-som-s2784-830.html
http://songkhoe.vn/dien-dan-viem-phoi-do-nam-s2533-1033.html
http://songkhoe.vn/hay-dung-lai-ngay-viec-dan-mat-vao-dien-thoai-boi-le-s2953-0-311820.html
http://songkhoe.vn/dien-dan-hoi-dap.html
http://songkhoe.vn/nguy-co-dot-quy-nao-do-gio-lam-viec-keo-dai-s2964-0-266576.html
http://songkhoe.vn/quy-dinh-su-dung.html
http://songkhoe.vn/bac-si-cong-tac-did49.html
http://songkhoe.vn/bac-si-cong-tac-did53.html
http://songkhoe.vn/bac-si-tu-van-lam-the-nao-de-be-ngu-het-dem-sau-va-ngon-giac-s2539-1131-311461.html
http://songkhoe.vn/dau-hieu-benh-nguy-hiem-tu-mong-tay-s2964-0-311784.html
帮我选择仅包含字符串“/chuyen-muc-”的字符串。
我尝试:
Elements links = doc.select("a[href]:contains(chuyen-muc-)");
但不工作
最佳答案
改用这个 CSS 查询:
a[href*=chuyen-muc-]
您可以使用正则表达式来更具体:
a[href~=chuyen-muc-(dinh|lam)]
上述 CSS 查询将只返回包含 chuyen-muc-dinh
或 chuyen-muc-lam
的链接。
关于java - Jsoup 解析/抓取包含特定字符串,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/36278094/
如本answer所述,如果浏览器不支持 e,可以设置后备游标。 G。 光标:抓取;。我现在的问题是获取这些图像。在我的驱动器上本地搜索“.cur”只给了我系统光标,其中 grab.cur 和 grab
以下代码在计算机上运行以从 Instagram 帐户中抓取数据。当我尝试在 VPS 服务器上使用它时,我被重定向到 Instagram 登录页面,因此脚本不起作用。 为什么当我在电脑上或服务器上时,I
我在使用 Ruby 和 Mechanize 将 POST 查询传递到站点的网站上。访问站点的查询基于 firebug,如下所示 param.PrdNo=-1¶m.Type=Prop¶m
我正在尝试抓取一个具有多个页面结果的网站,例如“1、2、3、4、5...”。 每个分页号都是到另一个页面的链接,我需要抓取每个页面。 到目前为止,我想出了这个: while lien = page.l
我正在使用 HtmlAgilityPack 在 C# Asp.Net 中执行 Scraping,到目前为止,我在从多个 Web 执行 Scratch 时没有遇到问题,但是,尝试弹出以下代码时出现错误
如果我有一个 css 文件做这样的事情 #foo:after{content:"bar;} ,有没有办法用 javascript 获取 :after 的内容?获取父元素的内容只返回 #foo 元素的内
问题是这样的: 我有一个 Web 应用程序 - 一个经常更改的通知系统 - 在一系列本地计算机上运行。该应用程序每隔几秒刷新一次以显示新信息。计算机仅显示信息,没有键盘或任何输入设备。 问题是,如果与
我想制作一个程序来模拟用户浏览网站和点击链接。必须启用 Cookie 和 javascript。我已经在 python 中成功地做到了这一点,但我想把它写成一种可编译的语言(python ide 不会
我制作了这个小机器人,它通过搜索参数列表进行处理。它工作正常,直到页面上有几个结果: product_prices_euros 给出了一半为空的项目列表。因此,当我与 product_prices_c
我需要找到一个单词的匹配项,例如: 在网上找到所有单词“学习”https://www.georgetown.edu/(结果:4个字)(您可以看到它按CTRL + F并搜索) 我有我的 Python 代
有一个站点\资源提供一些一般统计信息以及搜索工具的界面。这种搜索操作成本高昂,因此我想限制频繁且连续(即自动)的搜索请求(来自人,而不是来自搜索引擎)。 我相信有很多现有的技术和框架可以执行一些情报抓
这并不是真正的抓取,我只是想在网页中找到类具有特定值的 URL。例如: 我想获取 href 值。关于如何做到这一点的任何想法?也许正则表达式?你能发布一些示例代码吗?我猜 html 抓取库,比如 B
我正在使用 scrapy。 我正在使用的网站具有无限滚动功能。 该网站有很多帖子,但我只抓取了 13 个。 如何抓取剩余的帖子? 这是我的代码: class exampleSpider(scrapy.
我正在尝试从这个 website 中抓取图像和新闻 url .我定义的标签是 root_tag=["div", {"class":"ngp_col ngp_col-bottom-gutter-2 ng
关闭。这个问题需要更多focused .它目前不接受答案。 想改进这个问题吗? 更新问题,使其只关注一个问题 editing this post . 关闭上个月。 Improve this ques
我在几个文件夹中有数千个 html 文件,我想从评论中提取数据并将其放入 csv 文件中。这将允许我为项目格式化和清理它。例如,我在这个文件夹中有 640 个 html 文件: D:\My Web S
我在编写用于抓取网页的实用程序时遇到了一个问题。 我正在发送 POST 请求来检索数据,我模仿我正在抓取的网络行为(根据使用 fiddler 收集的信息)。 我已经能够自动替换我的 POST 中除 V
对于 Googlebot 的 AJAX 抓取,我在我的网站中使用“_escaped_fragment_”参数。 现在我查看了 Yandex 对我网站的搜索结果。 我看到搜索结果中不存在 AJAX 响应
我正在尝试抓取网站的所有结果页面,它可以工作,但有时脚本会停止并显示此错误: 502 => Net::HTTPBadGateway for https://website.com/id/12/ --
我是一个学习网络爬虫的初学者,由于某种原因我无法爬网this地点。当我在 Chrome 中检查它时,代码看起来不错,但是当我用 BeautifulSoup 阅读它时,它不再是可刮的。汤提到“谷歌分析”
我是一名优秀的程序员,十分优秀!