Multi-brand new/used car inventory search: an evolution story of python script to a web scraper to an API driven webapp

Tech Lounge
224

Hello Rusty,
Thank you for making this. I have downloaded the python and ran it as described in the readme. Unfortunately I don’t see any updates in any file. What am I missing?

2024-04-28 00:42:20 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{‘downloader/request_bytes’: 1315,
‘downloader/request_count’: 5,
‘downloader/request_method_count/GET’: 5,
‘downloader/response_bytes’: 967372,
‘downloader/response_count’: 5,
‘downloader/response_status_count/200’: 5,
‘elapsed_time_seconds’: 2.243731,
‘feedexport/success_count/FileFeedStorage’: 1,
‘finish_reason’: ‘finished’,
‘finish_time’: datetime.datetime(2024, 4, 28, 4, 42, 20, 545506, tzinfo=datetime.timezone.utc),
‘httpcompression/response_bytes’: 5467444,
‘httpcompression/response_count’: 5,
‘log_count/DEBUG’: 6,
‘log_count/INFO’: 12,
‘log_count/WARNING’: 1,
‘response_received_count’: 5,
‘scheduler/dequeued’: 5,
‘scheduler/dequeued/memory’: 5,
‘scheduler/enqueued’: 5,
‘scheduler/enqueued/memory’: 5,
‘start_time’: datetime.datetime(2024, 4, 28, 4, 42, 18, 301775, tzinfo=datetime.timezone.utc)}
2024-04-28 00:42:20 [scrapy.core.engine] INFO: Spider closed (finished)

225

Hey.

The python script was the scraper written by @coder8, whose thread then got merged with thread where I was building my app, which is web-based.

If you want to run his Python version, I’d send him PM. If you want to use my version to search for inventory, you can do it from web directly:

http://carscrapper.azurewebsites.net/

1 Like