No worries. Let me know if you need any help!
Yup, just quickly tested it on one Volvo site I have. Itās different URL than new cars, even easier. No need to look for image, and can be searched via query string parameters just like new inventory:
https://www.fallschurchdonbeyervolvo.com/demo-loaner-inventory.htm?model=XC40
Iāll add that next. I wonāt probably have time during this weekend, going on family fishing trip, so sometime next week.
Just took a look at some local dealers, and it seems quite a few have a similar URL. If anybody knows dealers with similar, a detailed list can probably be useful for the integration.
Have fun fishing!
Thanks and I second that, if someone can come up with dealer list that support this format, Iād just add them to config and these can be searched on. Minimum (if any) code tuning should be required.
Very few dealers have separate links for loaners. This link will only pull their storeās loaner inventory vs. all 3 Don Beyer Volvo stores inventory for new cars. So you need 3 separate loaner scrapers for each store.
Yeah, V60 or V60CC search will not return any V60CC. Has to be V60 Cross Country
I hear what youāre saying, but from feature delivery time perspective, if everything else is equal, itās easier for me to just add 3 config entries for DealerCom sites vs adding one DealerInspire superstore. DealerCom routine is much more generic and stable at the moment compared to DealerInspire, which is slowest and least reliable.
All 3 individual Don Beyer Volvo stores have dedicated links for loaners:
https://www.fallschurchdonbeyervolvo.com/demo-loaner-inventory.htm?model=XC40
https://www.dullesdonbeyervolvo.com/demo-loaner-inventory.htm?model=XC40
https://www.winchesterdonbeyervolvo.com/demo-loaner-inventory.htm?model=XC40
Making all that new inventory available to srapper engine is just adding 3 separate config entries for new inventory, same for loaners, when I add code for loaner support.
Yeah, I know. Whatever is easier, Iām all for it.
mbusa also has an api for dealer inventory here:
https://nafta-service.mbusa.com/api/inv/en_us/new/vehicles/search?model=GLC63C4&dealerId={some 5 digit code}&start=0&year=2020&sortBy=availability¤tDealerOnly=false
you need to use another api to get all dealer id though
Iām in. Not very good but time to put my CS degree to use
Had some time to quickly add loaner search for DealerCom Volvo sites. It only works on those for now.
Thatās awesome! The more the merrier. What languages/frameworks youād be able to help with coding?
So my fishing trip went kaput, but on a bright side Iād probably have some time over the weekend to sink into improving scrapper.
Volvo individual stores are pleasure to work with, they all seem to use same exact version of dealerCom platform, and have very consistent structure across all dealers I tried. I changed loaner detection from specific URL (not all dealers have it, just like @Ursus said), to a loaner image presence detection (as @volvo1 suggested, thanks!) and this detection is very reliable.
Added bunch of Volvo dealers, and currently weāre at 85 loaners/400 total new stock for XC90 for example:
These volvo dealers are basically plug and play, add entry to a config, get dealerās listings for loaners and new stock on UI.
Awesome It is usually a pain to find Volvo loaners on dealers sites (though they tend to be listed at the end, so one can start looking from last page)
That is awesome! Glad I could help out. Let me know if you need a list of dealers to add.
can you build a scraper for used cars?
You can. Same code, different URLs
Are you looking specifically for used cars at dealerships or everything? Though this is useful, there are a lot more used car dealerships with different website templates than new car dealers that use similar website templates.
I want to add dealers. How do I know if a dealer is Dealer.com, DealerOn or DealerInspire?
UPDATE: I believe I figured it out and submitted a pull request with a bunch of dealers.
I found 3 dealers that didnāt seem to belong to either Dealer.com, DealerOn or DealerInspire though. Are there other types out there? eg: Nabthat, DealerFIre
Hey thatās great⦠Iāve seen your pull request, Iāll review it ASAP. Gotta host 4th of July party, so probably early tomorrow. Happy 4th of July everyone.
Hey thanks for putting time and effort in helping with dealers. I looked at it quickly and everything looks good. I approved your pull and merged to master. Iāll double check everything tomorrow, to make sure weāre not missing anything, but so far it looks great.
Sorry, I should have posted earlier on how to identify DealerOn vs DealerInspire vs Dealercom website.
Go to a dealer website, run a search for new inventory for specific model and compare URL to those below. That should tell you which type of platform it is. If itās none of those, we donāt support it yet.
www.dealerwebsite.com/new-inventory/index.htm?model={model} - thatās dealerCom
www.dealerwebsite.com/searchnew.aspx?Model={model} - thatās DealerOn
www.dealerwebsite.com//new-vehicles/{model}/#action=im_ajax_call&perform=get_results&vrp_view=listview&page=1 - thatās dealerInspire
Substitute {model} with āX3ā for BMW for example.
Next morning edit:
So I reviewed all config changes, everything is top notch. Moved one dealer from DealerInspire to DealerOn, and thatās pretty much it⦠Donāt see results from Pacific BMW, but Iāll look at it later. Never mind, they are there, too much vodka last night lol.
Now we have huge inventory results for BMW, both east coast and west coast dealers across all 3 types:
768 new X3s. Search is becoming slower and slower, as expected, just under 3 minutes for example above. I guess we need to start thinking about adding regional filter. Should I go with the way regions are broken down here, on LH?
- [California]
- [Northeast]
- [Midwest]
- [South]
- [West]
Thatās good question, probably, or some just have their own custom built websites altogether. Adding new type of dealer for generic processing would require brand new crap engine routine though, and is not quick.
Ugh⦠Now with so many new BMW dealers, If you search for BMW across all 3 platforms at once, the search might timeout., depending on model. Apparently Azure has 230 seconds max hard timeout, that can not be changed. If operation takes longer than that, request will timeout. For now, donāt use all 3 dealer types at once for BMW searches, while Iām figuring out workaround. Try two, if it still times out, go one by one.
Gotta love Microsoft. Nothing is ever easy or straightforward