Australian car database

Out of the 100+ databases I made, Australia Car Database was the MOST DIFFICULT of all web scraping projects. Last release: 17 December 2019.

BIG database – download SAMPLES:
Australia Car Database SAMPLE 2019 edition (116 columns)
Australia Car Database SAMPLE original 2017 edition (96 columns)

Buy complete database + FREE updates for 1 year:

Small database – download SAMPLE:
Australia Make-Model-Year SAMPLE.xls

Buy complete database + FREE updates for 1 year:

Contact me for custom packages! For example you can ask for model naming, engine power and torque, wheels and tire dimensions. 1990-2019, 2005-2019 or whatever do you need. Price will be 0.3 eurocents / model.

Description & notes

First time made in June 2017 using Octoparse.com, a FREE scraper slow and buggy, had to run in small batches of 1000-2000 cars and took 2 weeks to complete project, 96 columns, without colors and features which were not scrap-able with Octoparse.

In mid-2017 source website removed META tags used by me to obtain Make and Model name separately from full car name, also removed a column highly appreciated by my customers: VIN. Due to this reason it was not worth my effort to re-scrap all cars, I was updating database every 3 months by scraping only last year of cars, manually splitting Make and Model, leaving older cars unchanged. I removed “trade-in prices” columns because it was variable over time.

In February 2019 I worked with an Australian student who made a Python script that took only 2 days to scrap all cars, gave me a BETA .py scraper with errors and Colors and Optional Features missing, which I reported and he fixed afterwards but instead of giving me final scraper, he setup scraper to run and update on monthly basis on his server and the agreement was to give me login info to download CSV. Everything looked perfect, his database included also Colors and Optional Features, but in April 2019 he became uncontactable and server was closed, leaving me and my customers with no updates.

In July 2019 I hired another Python programmer, an indian who impressed me saying that have over 80 databases created in Python, I gave him the BETA scraper from Australian student to fix errors. He turned to be a liar, very inexperienced, he used a stupid way to fix errors in Australian student’s scraper which caused more and more errors, wasting my time for 4 months testing his scraper and report errors, the idiot abused my lack of knowledge of Python + lack of time to check his code carefully, kept demanding extra $ over the price we initially agreed (NO legitimate programmer would charge extra money for fixing his own errors).

In September he offered to sell me his portfolio of 80+ databases for 100 EURO, which I paid, and in October when I was less busy and I checked them, I realized that at least 90% of them were open data available for free download on various sites (he was LYING that created himself). After providing 2 temporary updates with 2019 models in September and November, I started a fresh scrap of all 1960-2019 models in late November and published final version on 17 December 2019.

By buying Australia Full Specs database you will get 3 versions:

December 2019 version made by idiot indian which due to his unfixed typos 11 columns do not have data (is about columns with rare data available for less than 5% of cars), reason for which I offer also March 2019 update made by Australian student with all columns filled up. I also offer 1st version from June 2017 that include VIN column.

Shortly after Christmas when I was less busy, I checked his Python code in detail and spent 5 hours myself to fix all his errors that he did not fixed in 4 months (even if I never coded in Python before), the scraper is now filling up all columns except Colors and Optional Equipment (Indian programmer want another $100 for this and promise doing in 3 days. Are you fucking serious? Any skilled programmer wouldn’t spend more than 1 hour on this! I already waited for 4 months). I was planning to run scraper again all 100,000 cars when I find a skilled programmer to help me with codes for Colors and Optional Equipment, BUT…

In January 2020 the source website added a CAPTCHA that appear every time I access website using a new browser. The indian idiot, instead of apologizing for his errors and try to do better job next time, refuse to help me because I blamed his experience for errors done in 2019. I tried to ask help from other programmers but nobody was able to succeed. Anyone who know Python and want to give a try, please contact me!

There is possible source website to add additional anti-scraping measures in the future, requiring me to pay again and again a programmer to adapt scraper to continue updating this database in an automatic way, which will be a waste of money. Alternatively I can solve captcha in normal browser and save manually each car page in my computer then scrap locally, which will be a waste of time. In both cases, updating Australian database require a HUGE effort that will never be paid off so I might abandon this database. For comparison, American Year-Make-Model-Trim-Specs takes 5x less effort per update and its sales are 5x higher than Australian car database.

Full story and changelog: https://www.teoalida.com/cardatabase/australia-car-database-changelog/

List of 125 car makes included

Abarth, Alfa Romeo, Alpina, Alpine, Armstrong Siddeley, Asia Motors, Aston Martin, Audi, Austin, Austin Healey, Australian Classic Car, Bedford, Bentley, Bertone, Blade, BMC, BMW, Bolwell, Bufori, Bugatti, Buick, Cadillac, Caterham, Chery, Chevrolet, Chrysler, Citroen, Commer, CSV, Daewoo, Daihatsu, Daimler, Datsun, De Tomaso, Dodge, Elfin, Eunos, Ferrari, Fiat, Ford, Ford Performance Vehicles, Foton, FSM, Geely, Genesis, Giocattolo, Great Wall, Haval, HDT, Hillman, Hino, Holden, Holden Special Vehicles, Honda, Humber, Hummer, Hyundai, INFINITI, International, ISO, Isuzu, Jaguar, Jeep, Jensen, JMC, Kia, KTM, Lada, Lamborghini, Lancia, Land Rover, LDV, Lexus, Leyland, Lightburn, Lincoln, Lotus, Mahindra, Maserati, Maybach, Mazda, McLaren, Mercedes-Benz, MG, MINI, Mitsubishi, Morgan, Morris, Nissan, Noble, NSU, Opel, Pagani, Peugeot, Pontiac, Porsche, Proton, RAM, Rambler, Renault, Robnell, Rolls-Royce, Rover, Saab, Seat, Simca, SKODA, smart, SsangYong, Studebaker, Subaru, Suzuki, Tata, TD 2000, Tesla, Toyota, TRD, Triumph, TVR, Vanden Plas, Vauxhall, Volkswagen, Volvo, Wolseley, ZX Auto.

Data fields included

Percentages calculated for 1960-2017 database. If you buy 1990-2017, 2000-2017, etc, you will get higher completion ratio. Certain data fields are available for recent cars only, for example Fuel Consumption is available starting from 2000s.

Naming: Full car name 100%, ID 100%, Make 100%, Model 100%, Year 100%, Price 97.16%, Image URL 69.76%.

Description: Body 100%, Engine 99.94%, Transmission and Drivetrain 100%, Fuel Type 100%, Fuel Consumption 52.84%.

Overview: Badge 100%, Series 100%, Body 100%, No. Doors 99.97%, Seat Capacity 98.53%, Transmission 100%, Number of Gears 99.99%, Drive 99.99%, FuelType 100%, Recommended RON Rating 55.10%, Release Year 100%, VIN 77.14%, Country of Origin 99.97%, ANCAP Safety Rating 29.29%, Overall Green Star Rating 32.33%, Text Description 25.81%.

Engine: Engine Type 99.99%, Engine Location 97.77%, Engine Size 99.92%, Induction 99.94%, Engine Configuration 78.95%, Cylinders 99.93%, Camshaft 86.72%, Valves/Ports per Cylinder 86.19%, Compression ratio 74.18%, Engine Code 58.01%, Power 85.10%, Torque 81.67%, Power to Weight Ratio 81.42%, Acceleration 0-100km/h 37.14%, Maximum Speed 0.39%.

Fuel Fuel Type 100%, Fuel Capacity 78.03%, RON Rating 55.10%, Maximum Ethanol Blend 78.03%, Fuel Delivery 99.92%, Method of Delivery 99.90%, Fuel Consumption Combined 52.84%, Fuel Consumption Extra Urban 56.27%, Fuel Consumption Urban 58.10%, Fuel Average Distance (km) 52.61%, Fuel Maximum Distance 55.69%, Fuel Minimum Distance 57.33%, CO2 Emissions Combined 51.83%, CO2 Extra Urban 30.36%, CO2 Urban 30.40%, Greenhouse Rating 32.05%, Air Pollution Rating 32.07%, Green Star Rating 32.32%, Emission Standard 25.57%.

Dimensions and weight: Length 83.34%, Width 83.31%, Height 83.03%, Wheelbase 83.64%, Track Front 77.29%, Track Rear 77.28%, Tare Mass 61.18%, Kerb Weight 75.25%, Gross Vehicle Mass 58.79%, Gross Combination Mass 31.62%, Payload 46.69%, Boot Load Space Min 9.93%, Boot Load Space Max 10.96%, Towing Capacity braked 62.81%, Towing Capacity Unbraked 59.22%, Load Length 4.22%, Load Width 4.17%, Width Between Wheel Arches 2.28%.

Warranty: Warranty in Years from First Registration 72.28%, Warranty in Km 70.45%, Warranty Customer Assistance 47.05%, Warranty Anti Corrosion from First Registration 13.23%, Free Scheduled Service 0.98%, First Service Due in Km 42.11%, First Service Due in Months 34.40%, Regular Service Interval in Km 43.86%, Regular Service Interval in Months 39.66%.

Steering and Wheels: Steering 65.04%, Rim Material 39.66%, Front Rim Description 79.46%, Rear Rim Description 79.46%, Front Tyre Description 75.54%, Rear Tyre Description 75.55%.

How often do you want updates? (Australia car database)

Every month
Every 3 months
Every 6 months
Every year
One-time project, not interested in updates
Other
Please Specify:
Created with Poll Maker

20 thoughts on “Australian car database

  1. Hello

    I’m looking for Australian car specification data from 1970 to current year

    Database should contain following fields

    Make, Model,Year, Body Type, Transmission and Variant

    Thanks

    1. 2 comments in same day…. S and J (full names removed at request), are you from same team?

      A week ago another 2 people contacted me via LIVE CHAT about Australian car database, so I decided to start experimenting various scraping methods on various sites.
      C-TEST.xls (link removed)
      R-TEST.xls (link removed)
      I can sell you specific fields only.

      carsales.com.au have used cars classifieds only and no specs for every car sorted by make and model. So it’s useless

      1. 2 comments in same day…. S and John Romaine, are you from same team?

        No mate.

        Coincidence lol

        Let me have a look at your XLS files.

  2. Hi .
    Hi.
    I would interested in the Australian DB but I am just doing something for fun so no commercial intentions. I could pay something but not much. Carsales does have specs in another tab. Not sure about the alpha order. I would be also interested in the global one. But I just want basic info. What price ? 40 euros ?
    Thanks,
    Frank.

    1. I can’t believe how many comments from Australia I get in a short period of time, this page being here since May 2016 so for 1 year I did not got a single comment!

      I noticed that CarSales.com.au and CarPoint.com.au have a Research section, they not just show same data like Redbook but are also same pain in the ass at scraping, my partner scraper takes 50 seconds per page and jumps some pages (which at other websites takes max 2 seconds per page and up to 5 pages per second). So I use a third-party scraper which takes about 10-15 seconds per page = takes over 300 hours to scrap, but I run 4 scraps at same time thus reduce time to 7 days.

      For global database with basic info only you can buy https://www.teoalida.com/cardatabase/car-models-list/

      1. Hi.

        Its a coincidence that I just happened to have some spare time and wanted to do a little project. I got laid off from working at Oracle for 26 years. While I know a bit about DBs I dont know much about scraping.

        Well I assume the time effects the cost. I just want basic data. Any odeas on the prices I asked you for ?
        .btw where are you located ?

        Cheers,
        Frank.

  3. Than you to all people who showed interest in this project encouraging me to start building Australian car database, 2 of them contacted me via LIVE CHAT on 26 May, then S and John Romaine who left comments on 3 and 4 June.

    Database finalized and published today!

    Feel free to contact me and negotiate price if you want a specific set of columns and historical coverage!

  4. After months of trying to gather a Australian Car database I found teoalida’s website. At the time he didn’t have a Australian database but was ready for the challenge.
    After downloading an checking the database today, it is extremely detailed, complete and im extremely happy with the purchase. If anyone else is looking for this database, it wont disappoint.

    Thanks again for your help mate.

  5. Hi

    I am looking for an Australian vehicle database and came across your website. I have a looked at your sample database. It has more information than I need, but at the same time it is missing some critical information. I need a database that will have the following information:
    – overall length
    – overall width (no mirrors)
    – overall height
    – wheelbase
    – front overhang
    – rear overhang
    – turning radius/diameter – wall to wall
    – turning radius/diameter – kerb to kerb
    – ground clearance (unladen)
    – ground clearance (fully laden)
    – front or rear axle lock to lock time
    – front truck width
    – rear track width
    – front tire width and diameter
    – rear tire width and diameter
    – approach angle (front underside of vehicle)
    – departure angle (rear underside of vehicle)
    – dimensions of side mirrors (including any attachments so it can fixed to the body of the vehicle).

    I realise the above list may be quite extensive. It is not necessary to have all the information, but it would be good to have. Order of importance is top to bottom. To give you a better idea our need for the database, we work as consulting traffic engineers and we design car parks. From time to time, we have to assessed whether a car park will be practicable or not. We need to run vehicle path simulation using vehicle dimensions. The software we used requires the above as input.

    Can you give me a quote to create such a database?

    I assume the data you collected relates to vehicles available for sale in Australia, not necessary registered vehicles? Are you able to create database for registered vehicles.

    1. I already included all possible data that could be scraped from the source website. If you need other data you should suggest another website to scrap data from. If you want a custom package with just few columns I can sell you at lower price than full specs.

        1. Probably 0.3 cents per car, select exact columns you want. Use live chat in lower-left corner of screen so we can communicate faster.

  6. Hi there, I need a custom database. Can you please provide a quote for the Australian car database between 1960-2018 with the following:

    Full car name, make, model, year, body description.

    Thanks,

    Ryan

  7. Hello I’m looking for a lw16r tractor it was stopped being made around the 1960s and we have one our selves left on a property we bought. I would like to get it going again and I need to know more about it because when I google it it comes up with the first model lw12. The company who makes them is LeTourneau’s Westinghouse and was manufactured in Rydalmere in New South Wales, Australia. can you please find out a website or any information you would have on them thanks George

    1. Do you have any problem with chat box? You left the chat after short time.

      Chat started on Saturday, December 22, 2018 11:49:11 AM

      (11:49:11 AM) *** Visitor 88341860 has joined the chat ***
      (11:49:13 AM) Teoalida (site owner): Hi there… are you looking for a specific database? Let me know if I can help you!
      (11:50:25 AM) Visitor 88341860: Hello I’m trying to find information about the lw16r tractor but it was made in New Zealand wondering if you have information on it
      (11:50:52 AM) *** Teoalida has joined the chat ***
      (11:51:10 AM) Teoalida: I do not have any database with tractors right now, but I can create one scraping data from https://www.ritchiespecs.com/
      (11:51:18 AM) *** Visitor 88341860 has left ***

  8. Hi,
    We are currently looking for a quality Auto Data base supplier, just wondering how up to date are you data files, with respect of course,

    Australian needed to start with,

    Other countries will be required after the initial order & purchase.

    Looking forward to your reply.

    Kind Regards
    Anthony

Leave a Reply

Your email address will not be published. Required fields are marked *