javascript - Promise 并将数组中的元素添加到另一个数组中的对象中

Question

该作业是一个命令行 Node 应用程序，它从特定站点上抓取一些数据并将该数据保存到 CSV 文件中。

我正在使用scrape-it抓取数据并成功获取我需要的所有数据，但我正在努力弄清楚如何将每个 URL(存储在 urls 中)添加到其相应的衬衫对象中，该对象是一个对象数组。

这是我到目前为止所拥有的。

const scrapeIt = require("scrape-it");

const mainURL = "http://shirts4mike.com/";

scrapeIt(`${mainURL}shirts.php`, {
  pages: {
    listItem: ".products li",
    name: "pages",
    data: {
      url: {
        selector: "a",
        attr: "href"
      }
    }
  }
})
  .then(({ data }) => {
    const urls = data.pages.map(page => `${mainURL}${page.url}`);
    console.log(urls);
    const shirtCalls = urls.map(url =>
      scrapeIt(url, {
        name: {
          selector: ".shirt-picture img",
          attr: "alt"
        },
        image: {
          selector: ".shirt-picture img",
          attr: "src"
        },
        price: {
          selector: "span.price"
        }
      })
    );
    return Promise.all(shirtCalls);
  })
  .then(shirtResults => {
    const shirts = shirtResults.map(shirtResult => shirtResult.data);
    console.log(shirts);
  });

所以“衬衫”给我的输出是，

[ { name: 'Logo Shirt, Red',
    image: 'img/shirts/shirt-101.jpg',
    price: '$18' },
  { name: 'Mike the Frog Shirt, Black',
    image: 'img/shirts/shirt-102.jpg',
    price: '$20' },
  { name: 'Mike the Frog Shirt, Blue',
    image: 'img/shirts/shirt-103.jpg',
    price: '$20' },
  { name: 'Logo Shirt, Green',
    image: 'img/shirts/shirt-104.jpg',
    price: '$18' },
  { name: 'Mike the Frog Shirt, Yellow',
    image: 'img/shirts/shirt-105.jpg',
    price: '$25' },
  { name: 'Logo Shirt, Gray',
    image: 'img/shirts/shirt-106.jpg',
    price: '$20' },
  { name: 'Logo Shirt, Teal',
    image: 'img/shirts/shirt-107.jpg',
    price: '$20' },
  { name: 'Mike the Frog Shirt, Orange',
    image: 'img/shirts/shirt-108.jpg',
    price: '$25' } ]

但是我想要得到的最终结果是......

[ { name: 'Logo Shirt, Red',
    image: 'img/shirts/shirt-101.jpg',
    price: '$18',
    url: 'http://shirts4mike.com/shirt.php?id=101' //which is at urls[0]
  },
  { name: 'Mike the Frog Shirt, Black',
    image: 'img/shirts/shirt-102.jpg',
    price: '$20',
    url: 'http://shirts4mike.com/shirt.php?id=102' //urls[1]
  }, //...etc etc

希望一切都有意义，对于 promise (和 Node )来说仍然很新，所以我感觉有点超出了我的深度。预先感谢您!

Answer 1

尝试这样的事情:

const scrapeIt = require("scrape-it");

const mainURL = "http://shirts4mike.com/";

scrapeIt(`${mainURL}shirts.php`, {
  pages: {
    listItem: ".products li",
    name: "pages",
    data: {
      url: {
        selector: "a",
        attr: "href"
      }
    }
  }
})
  .then(({ data }) => {
    const urls = data.pages.map(page => `${mainURL}${page.url}`);
    console.log(urls);
    return urls.map(async (url) => {
      let urlObj = await scrapeIt(url, {
        name: {
          selector: ".shirt-picture img",
          attr: "alt"
        },
        image: {
          selector: ".shirt-picture img",
          attr: "src"
        },
        price: {
          selector: "span.price"
        }
      });

     return {...urlObj.data, url};
   });
  })
  .then(shirtResults => {
    console.log(shirtResults);
  });