IT.COM

NameBio extraction

Spaceship Spaceship
Watch

gdndd

Established Member
Impact
225
Hi,

Is it possible to extract all domain sales from Namebio in .CSV or .XLSX ?

Thanks
 
0
•••
The views expressed on this page by users and staff are their own, not those of NamePros.
Hi,

Is it possible to extract all domain sales from Namebio in .CSV or .XLSX ?

Thanks

Probably but it's certainly against their ToS. There are easier ways to get the information. I have every sale dating back to the early 90s. ;)
 
0
•••
ho ... And Are you sharing this information ???
 
0
•••
1
•••
Hum ... OK ! Then thank you for making jealous,,,
 
1
•••
Hum ... OK ! Then thank you for making jealous,,,

You'll find the answer but don't break any rules to get it. It'll come back to get you. I like where your mind is though. Data is king and to become a king (not saying I am by any means) in this industry you must invest both time and money. It'll pay in spades but the cost is definitely upfront.
 
6
•••
I think DNJournal also good one to check past sales
 
1
•••
I believe many domain price history sites including NameBio only provide a limited amount of reported sales but also have a large amount of data for sale. Just inquire, but expect to pay up for it.
 
1
•••
Once I read somewhere that only 50% of domain sales are reported taking into consideration all websites which report sales. The remaining 50% goes private.
 
0
•••
In term of transactions, namebio (and all others) misses a lot of transactions. Much more than 50% IMHO:

- any transaction below 100$: there are probably a lot of them in number (not in $ volume)

- direct sales. My estimate is that namebio records may be ~10% of expensive transactions (say above 10k) and less than ~10% of geo domain transactions (and more generally sales where end-users are easy to identify such as JonDoe.com (FirstNameLastName) )

Yet, it's a very valuable tool to see what's valuable.
 
1
•••
- direct sales. My estimate is that namebio records may be ~10% of expensive transactions (say above 10k) and less than ~10% of geo domain transactions (and more generally sales where end-users are easy to identify such as JonDoe.com (FirstNameLastName) )


- Expensive transactions are the kingdom of domainers and broker who dealing directly with and users.
 
0
•••
Hi,

Is it possible to extract all domain sales from Namebio in .CSV or .XLSX ?

Thanks

We don't hide the prices in images any more, so you can just copy and paste the results directly into Excel. Set your results per page to All (i.e. 100) to make it faster. If you mean all as in everything that matches your search criteria than no, that isn't possible at the moment. Not easily anyway, you can divide your search into price bands to get more results.

I believe many domain price history sites including NameBio only provide a limited amount of reported sales but also have a large amount of data for sale. Just inquire, but expect to pay up for it.

We don't sell our data at this time, but we get asked that a lot. As Shane alluded to, there is a place you can buy another site's data for a four-figure sum. Since before I joined NameBio there was functionality in place to detect scrapers and feed them fake sales instead of blocking access. Their records are really dirty and have lots of completely made up sales. Enjoy your "data" Shane :)

Once I read somewhere that only 50% of domain sales are reported taking into consideration all websites which report sales. The remaining 50% goes private.

There's no way to know, any number someone says is entirely made up. I assume the data we collect is just the tip of the iceberg in terms of the total market, but it does represent a high percentage of the wholesale market which is all that can be tracked at scale anyway.

In term of transactions, namebio (and all others) misses a lot of transactions. Much more than 50% IMHO:

- any transaction below 100$: there are probably a lot of them in number (not in $ volume)

- direct sales. My estimate is that namebio records may be ~10% of expensive transactions (say above 10k) and less than ~10% of geo domain transactions (and more generally sales where end-users are easy to identify such as JonDoe.com (FirstNameLastName) )

Yet, it's a very valuable tool to see what's valuable.

We record sales under $100, we just don't load them into the site. We may eventually if enough people want to see them. Personally I think the volume of these low dollar sales is just too high, and the research value of them is too low, to justify the added "cost" of keyword parsing, categorization, query speed, etc.

Take for example the last 30 days of GoDaddy expired auctions, we added 4,237 sales for a total of nearly $2 million. If we added sales under $100 we would have loaded an additional 14,332 sales but only added about $400k in dollar value. Do people really want tens of thousands of $12 sales cluttering the results? Most people are just going to end up setting the min price to $100 anyway so the low dollar sales don't push the sales they care about out of the 100 result limit.

I'm curious where your 10% estimate comes from. Is that in reference to private transactions? Because of the venues we track we get an extremely high percentage of their sales, some we capture literally everything. There's no way to know how many sales happen in private. But with bulk sales and venues we don't track like Afternic, SnapNames, the forums, truly private sales from people contacting the original registrant, etc. you very well could be right.
 
9
•••
Enjoy your "data" Shane :)

Harsh, bro.

Do you really think I didn't check my "wikipedia" style encyclopedia against the real thing?

;)
 
0
•••
Harsh, bro.

Do you really think I didn't check my "wikipedia" style encyclopedia against the real thing?

;)
I think if you had the ability to cross reference a million records against our site without triggering our anti-scraping measures and without showing up in our logs you wouldn't have paid them several grand for "their" data in the first place.

That would have been an ambitious undertaking without having known anything was amiss. Plus it wouldn't really tell you anything because they legitimately capture some sales that we miss, so not being in our database doesn't mean it wasn't a real sale. Can't really be sure which it is.

I was just breaking your balls man, didn't mean it in a harsh way.
 
0
•••
I think if you had the ability to cross reference a million records against our site without triggering our anti-scraping measures and without showing up in our logs you wouldn't have paid them several grand for "their" data in the first place.

That would have been an ambitious undertaking without having known anything was amiss. Plus it wouldn't really tell you anything because they legitimately capture some sales that we miss, so not being in our database doesn't mean it wasn't a real sale. Can't really be sure which it is.

I was just breaking your balls man, didn't mean it in a harsh way.

I know. I'm just playing anyways. :)

My only point is, unless you stuff an exceptional amount of false records, the data is still statistically relevant. If you changed the records to similar values to avoid detection, it's still statically relevant. Further, anyone who knows how to analyze bulk data will be able to throw out obvious anomalies.

While I don't condone their actions, as you've certainly put up the time and money to offer a far superior service, it is our only option if we want the data.

I'm surprised you don't sell the data on an individual and contractual basis. A simple non-compete would be sufficient to protect yourself from further competition. Even still, I would argue that increased competition would still not be enough to overtake your dominance within the market.

At the end of the day, if it were available, I would chose your data over anyone else's.

Just my 2 cents.
 
1
•••
Do people really want tens of thousands of $12 sales cluttering the results?
No, but the form already has a price range so the first one can simply default to $100, then anyone who wants to change it can and everyone wins.

Alternatively, have a separate search area dedicated for "domains sold for $100 and under".

I think there will be plenty of people wanting to study sales under $100. Some operate in this price band based on their business model and/or local economical climate etc, and many will make use of knowing what certain domains are bought for which sold for a higher profit.

If you already have all this data in the DB then it's a shame to not utilise it :)


Also, if you create a separate dedicated "under $100" area, then different ads than (eg) Escrow to target domainers who trade in under $100 could be lucrative for you too.
 
5
•••
We don't sell our data at this time

Sorry Michael, I wrote:

I believe many domain price history sites including NameBio only provide a limited amount of reported sales but also have a large amount of data for sale. Just inquire, but expect to pay up for it.

I got NameBio confused with DNPric.es. I thought I remembered seeing a sales page at NameBio, but it was not your site after all.

http://dnpric.es/download/
 
1
•••
I like too the idea of having transactions below 100$: it wopuld help finding what does NOT sell and explore different business models.
 
1
•••
We don't hide the prices in images any more, so you can just copy and paste the results directly into Excel. Set your results per page to All (i.e. 100) to make it faster. If you mean all as in everything that matches your search criteria than no, that isn't possible at the moment. Not easily anyway, you can divide your search into price bands to get more results.

Nice tips!

We don't sell our data at this time, but we get asked that a lot.

Seems like a reasonable thing to do. Do you not want to worry about accepting payments?
 
0
•••
0
•••
0
•••
@Michael - If we do a lot of MANUAL research on your site could we unintentionally trip your scraping detection and in turn be looking at false data?

In other words I mean physically looking manually at thousands of records in a systematic way, while in no way scraping your data, just for educational purposes (looking for trends).

I have been known to spend a lot of time on NameBio, sometimes using some pretty unconventional means (compared to a normal end user) to systematically work my way through more than 100 results in a pattern.

I can understand you having this measure in place, but I wonder if innocent people are ever fed false data?

Thanks,
Mike.
 
0
•••
We haven't returned false data as a response to scraping since I re-launched the site at the beginning of last year. While it is funny, I had concerns for exactly the reasons you mentioned. That comment was in reference to buying data from certain other sites who have stolen what we collected in the past, their data is definitely bad. I think this was only done after manual review though, not automatically.

What we do now is throttle/ban, and people can contact us in the very rare event that they triggered something manually. I'll occasionally change a sale price here and there by a few bucks to see who is publishing scraped data, but I don't do this often, usually just in response to seeing something suspicious.

I don't want to get too much into our algorithms for detecting scraping, but I will say it would require some pretty unusual activity and has nothing to do with volume. I don't consider trying to work around the 100 result limit to be suspicious, I've actually suggested ways for people to get around it because I hate it. It's only in place because it makes it very difficult to get every sale in the entire database without doing some really weird searches for a prolonged period of time.

I am aware of DNBolt, they got a big chunk of the data before we had any algorithms in place and we were just doing basic throttling. Then I noticed and started putting in some advanced detection measures, and we went back and forth for a few months one-upping each other. He's very clever, but as it stands I would be pretty impressed if someone could get all of the data, and we haven't accidentally blocked anyone in a while who wasn't trying to grab all of the sales.

All that said Adam and I both believe that data should be free and open, the anti-scraping is mostly about protecting quality of service and reducing costs. Having to support hundreds of thousands of bot requests can slow down the site or increase our hosting costs. What we'd really like to do it just make the data available for download, but once the data is out there it can never be undone, so we're taking our time with that decision.

If you ever get blocked shoot me a PM with your IP, or submit our contact form, and I'll take care of it. I'm also happy to help with custom data requests when I can.
 
6
•••
Back