Web Analytics Made Easy -
StatCounter

Australian car database

Small database – see SAMPLE

BIG database – see SAMPLE

Contact me for custom packages! For example you can ask for model naming, engine power and torque, wheels and tire dimensions. 1990-2019, 2005-2019 or whatever do you need. Price will be 0.3 eurocents / model.

Australia car database

I am making vehicle databases for Europe since 2003, for America since 2014, for India since 2015… what is next to do in 2016? Maybe Australia?

I created this page in May 2016 for marketing experiment to see how many people view Australia page and decide whenever worth my effort to create a car database for a country with such small population which may translate in just few sales of automobile database per year.

The page got very low traffic for one year, but during last days of May 2017 I got 2 people interested in purchasing an Australian car database, so I started studying possible sources of data, another 2 people left comments on 3 and 4 June, they asked me to scrap data from several possible websites (despite of the legal issues of scraping – HAD TO DO IT to serve people who had no other choice).

Due to anti-scraping measures on the source website, my programmer partner’s scraper don’t work so I had to look online for another scraper and found one slow and buggy, crashing a lot, forcing me to run it in small batches of about 1000-2000 cars and assemble them, scraping took about 2 weeks, and published database on 16 June 2017. From the initial 4 customers, only 1 ended purchasing, but new customers came and the number of sales in first months has been surprisingly high for a country with just 1/13 population of United States.

In November 2017, first customer asked for an update, and when tried to do it, I noticed that the source website removed META tags for year, make, model, so the only place where these essential information are displayed is page title, year, make, model, and badge are in a single field, requiring me to manually separate them after each update. Update takes 1-2 days and involving scraping last year of cars only. I had a huge luck to scrap all 90000+ cars in June 2017 to get make, model, year separately, automatically.

At April 2018 update the source website removed several data fields such as VIN, added them back few days later, removed again after 1 month, etc.

In February 2019 I met a student from Australia who asked few questions about my business, I answered them thinking that he is a customer looking to buy a database, but turned a student and later he threatened me that will build his own car database selling website if I don’t help him (help with what?), he claim that scraped from ******* with Python. ,but we decided to work together rather than competing each other. He also said that if I have more customers interested in web scraping I can pass to him, and I did passed one who probably paid him another $500.

He gave me a BETA .py scraper which was not working (I paid for it), I told him to fix and after few days he came with idea to host scraper on AWS and let script run automatically on schedule, he gave me me username/password where I could export data as CSV and sell via my website. I did this, providing an update for all my customers on 24 March (118 columns) and promising to all customers monthly updates that involve re-scraping ALL 1960-present cars, not just latest year, Private price guide and Trade in price guide will be updated accordingly.

2 customers reported missing data for most cars in Standard / Optional Equipment columns. I asked student to fix errors, on 18 April 2019 he replied “I’ve been really busy lately with Uni exams, interviews for jobs and the insurance project. Sorry Ill get to the Australian database as soon as I can” and that was the LAST day I heard from him. He did not signed in anymore. AWS account was suspended, probably because AWS offer free service and bill you at end of month, and he did not paid bills, and all what I had was a non-working BETA scraper. I think that he may also have died in car crash, he told me that passed driving exam recently and given by the fact that his dad owned a business, he may have got a very powerful car.

While I can update database again using my old method (96 columns only, without average trade-in value, without colors and features, etc), adding new cars and not updating older records, I hope to find another Python expert to correct his scraper and make it running properly.

I found someone in India experienced in Python, I paid him $50 and $100 for two small projects that he done successfully, then in July 2019 I gave him the BETA scraper from Australian student and we agreed $250 to fix it… he said that it is the most difficult project he ever done in Python, he also made many errors, we hardly met online at same time, only in September 2019 I can say that he fixed most errors after paying another $150 and I can extract most of data from the source website. I decided to provide you a TEMPORARY update with 2019 models beside the March 2019 update 1960-2019, and I ask all my customers to report errors so I can tell programmer to fix them before scraping all 1960-2019 cars with this new scraper, so whole database will be harmonized.

List of car makes included

Abarth, Alfa Romeo, Alpina, Alpine, Armstrong Siddeley, Asia Motors, Aston Martin, Audi, Austin, Austin Healey, Australian Classic Car, Bedford, Bentley, Bertone, Blade, BMC, BMW, Bolwell, Bufori, Bugatti, Buick, Cadillac, Caterham, Chery, Chevrolet, Chrysler, Citroen, Commer, CSV, Daewoo, Daihatsu, Daimler, Datsun, De Tomaso, Dodge, Elfin, Eunos, Ferrari, Fiat, Ford, Ford Performance Vehicles, Foton, FSM, Geely, Giocattolo, Great Wall, Haval, HDT, Hillman, Hino, Holden, Holden Special Vehicles, Honda, Humber, Hummer, Hyundai, Infiniti, International, ISO, Isuzu, Jaguar, Jeep, Jensen, JMC, Kia, KTM, Lada, Lamborghini, Lancia, Land Rover, LDV, Lexus, Leyland, Lightburn, Lincoln, Lotus, Mahindra, Maserati, Maybach, Mazda, McLaren, Mercedes-Benz, MG, MINI, Mitsubishi, Morgan, Morris, Nissan, Noble, NSU, Opel, Pagani, Peugeot, Pontiac, Porsche, Proton, RAM, Rambler, Renault, Robnell, Rolls-Royce, Rover, Saab, Seat, Simca, Skoda, smart, SsangYong, Studebaker, Subaru, Suzuki, Tata, TD 2000, Tesla, Toyota, TRD, Triumph, TVR, Vanden Plas, Vauxhall, Volkswagen, Volvo, Wolseley, ZX Auto.

Data fields included

Percentages calculated for 1960-2017 database. If you buy 1990-2017, 2000-2017, etc, you will get higher completion ratio. Certain data fields are available for recent cars only, for example Fuel Consumption is available starting from 2000s.

Naming: Full car name 100%, ID 100%, Make 100%, Model 100%, Year 100%, Price 97.16%, Image URL 69.76%.

Description: Body 100%, Engine 99.94%, Transmission and Drivetrain 100%, Fuel Type 100%, Fuel Consumption 52.84%.

Overview: Badge 100%, Series 100%, Body 100%, No. Doors 99.97%, Seat Capacity 98.53%, Transmission 100%, Number of Gears 99.99%, Drive 99.99%, FuelType 100%, Recommended RON Rating 55.10%, Release Year 100%, VIN 77.14%, Country of Origin 99.97%, ANCAP Safety Rating 29.29%, Overall Green Star Rating 32.33%, Text Description 25.81%.

Engine: Engine Type 99.99%, Engine Location 97.77%, Engine Size 99.92%, Induction 99.94%, Engine Configuration 78.95%, Cylinders 99.93%, Camshaft 86.72%, Valves/Ports per Cylinder 86.19%, Compression ratio 74.18%, Engine Code 58.01%, Power 85.10%, Torque 81.67%, Power to Weight Ratio 81.42%, Acceleration 0-100km/h 37.14%, Maximum Speed 0.39%.

Fuel Fuel Type 100%, Fuel Capacity 78.03%, RON Rating 55.10%, Maximum Ethanol Blend 78.03%, Fuel Delivery 99.92%, Method of Delivery 99.90%, Fuel Consumption Combined 52.84%, Fuel Consumption Extra Urban 56.27%, Fuel Consumption Urban 58.10%, Fuel Average Distance (km) 52.61%, Fuel Maximum Distance 55.69%, Fuel Minimum Distance 57.33%, CO2 Emissions Combined 51.83%, CO2 Extra Urban 30.36%, CO2 Urban 30.40%, Greenhouse Rating 32.05%, Air Pollution Rating 32.07%, Green Star Rating 32.32%, Emission Standard 25.57%.

Dimensions and weight: Length 83.34%, Width 83.31%, Height 83.03%, Wheelbase 83.64%, Track Front 77.29%, Track Rear 77.28%, Tare Mass 61.18%, Kerb Weight 75.25%, Gross Vehicle Mass 58.79%, Gross Combination Mass 31.62%, Payload 46.69%, Boot Load Space Min 9.93%, Boot Load Space Max 10.96%, Towing Capacity braked 62.81%, Towing Capacity Unbraked 59.22%, Load Length 4.22%, Load Width 4.17%, Width Between Wheel Arches 2.28%.

Warranty: Warranty in Years from First Registration 72.28%, Warranty in Km 70.45%, Warranty Customer Assistance 47.05%, Warranty Anti Corrosion from First Registration 13.23%, Free Scheduled Service 0.98%, First Service Due in Km 42.11%, First Service Due in Months 34.40%, Regular Service Interval in Km 43.86%, Regular Service Interval in Months 39.66%.

Steering and Wheels: Steering 65.04%, Rim Material 39.66%, Front Rim Description 79.46%, Rear Rim Description 79.46%, Front Tyre Description 75.54%, Rear Tyre Description 75.55%.

List of updates

122 makes, 92885 cars, as 16 June 2017. Initial launch.
123 makes, 93901 cars, as 20 November 2017. After about 10 sales, saw first person who ask for an update.
123 makes, 96604 cars, as 20 April 2018 I planned to update every 3 months, but the source website wasn’t working in February and March, thus only in April been able to do the update.
123 makes, 97610 cars, as 30 August 2018.
123 makes, 98225 cars, as 10 December 2018.
127? makes, 99925 cars, as 24 February 2019, a test with new scraper, not published.
124 makes, 100492 cars, as 24 March 2019, published with 118 columns instead of 96.
3002 cars (2019 models only), temporary update as 18 September 2019.

How often do you want updates? (Australia car database)

Every month
Every 3 months
Every 6 months
Every year
One-time project, not interested in updates
Other
Please Specify:
Created with Poll Maker

20 comments

3 pings

Skip to comment form

    • Anthony on February 4, 2019 at 9:00 am
    • Reply

    Hi,
    We are currently looking for a quality Auto Data base supplier, just wondering how up to date are you data files, with respect of course,

    Australian needed to start with,

    Other countries will be required after the initial order & purchase.

    Looking forward to your reply.

    Kind Regards
    Anthony

    • Eric on April 24, 2019 at 4:14 am
    • Reply

    Looking forward to working with this; thanks for your assistance so far!

  1. […] Australia […]

Leave a Reply

Your email address will not be published.