Need Suggestions - Finishing up making a domain name extracting firefox extension.

RegFee · 03-26-2008, 03:07 PM

I've created a link extracting/domain filtering firefox extension which is almost near completion. Here's a screenshot:

So far it does the following:

Every time the page changes, it scans for links (<a href="....">) and cleans the domain names out of the list.
Then it removes all duplicate entries.
Finally, if the list gets to a preset length, it basically cuts and pastes the list into a text file.

Some cool things I've done with it:

Combined with an auto-surfing bot extension, the tool harvests domain names from around the internet by searching google for a predetermined keyword, then following all of the links.
Going through digg.com (or any site) archives and find available names
Visiting link directories and just surfing around, collecting potentially available domain names (with at least one backlink, obviously)
Just leaving it running while doing normal everyday surfing.

Here's some things I've considered adding:

Real time availability checking - not really practical since I'll have to get my own domaintools api license ($$$$) or require users to get their own.
Option to send list to registrar for avail. check (ie Moniker bulk script) - got this working but it opens in the sidebar instead of the main window. Since this is my first major javascript project, I'm just going to leave this feature off for now rather than rack my brain.
Option to change how many domains get harvested before sending to file.
Option to not save domains to a file in the first place - necessary because javascript pops up the standard "are you sure you want to give this script access to save files?" warning and people probably won't feel comfortable with that.
Do away with the sidebar altogether and make it run in the statusbar, saving domains as it goes along. The sidebar can't be closed while running a script, so it reduces the visibility.

Ok, what I need help with:

First off, I just need to know if there is any public interest in this.
Secondly, if you are interested, which features would make it better for you.

Sorry I don't have a beta version yet. I've got to clean the code up a bit.
Thanks!
-Jorge

Spade · 03-26-2008, 04:38 PM

Jorge,

Sounds like a really cool project. Look forward to seeing what you come up with in the end.

Justin

CharlieA · 03-26-2008, 11:53 PM

Very nice. I hope you get this completed.

netfleet · 03-27-2008, 12:18 AM

Sounds great! Would love to beta test!

How about also harvests URLs that people type into their address bars? Obviously as domainers, we'd naturally check availability of any domain we happened to type in that didn't resolve but would be great to have something that automatically stores these type-ins for each computer.

Then you'd just install it on each computer in your internet cafe & gain some really valuable info!

Is this possible?

RegFee · 03-27-2008, 12:40 AM

Originally Posted by netfleet

Sounds great! Would love to beta test!

How about also harvests URLs that people type into their address bars? Obviously as domainers, we'd naturally check availability of any domain we happened to type in that didn't resolve but would be great to have something that automatically stores these type-ins for each computer.

Then you'd just install it on each computer in your internet cafe & gain some really valuable info!

Is this possible?

I'm sure it'd be possible to detect if a page didn't resolve, do an availability check, and log the domain name. Unfortunately, I think there would be a bit of a privacy issue there.

Once I've cleaned up the gui a bit (as well as the code, I really need to learn to nest from the beginning) I'll definitely release a beta for NPers. It will probably be in beta for a while though, since this is just a between-classes-got-a-few-minutes-to-kill project. But hey, gMail is still in "beta" so I've got nothing to worry about, right?

????: NamePros.com http://www.namepros.com/domain-name-discussion/449536-need-suggestions-finishing-up-making-domain.html

Thanks!
Jorge

xrvel · 03-27-2008, 01:14 AM

Nice. What about sorting the domains alphabetically.

- Kurniawan.

bobdigital · 03-27-2008, 08:29 AM

Are you searching for expired domains that are linked to other websites?

RegFee · 03-27-2008, 12:02 PM

Originally Posted by xrvel

Nice. What about sorting the domains alphabetically.

- Kurniawan.

I kind of like the idea of keeping it sorted by the order in which the names were grabbed, that way you can get an idea of which names you got off the current page. However, nothing says I can't add it as an option. Speaking of which, I've managed to add an options window now. Just gotta figure out how to implement those options

Originally Posted by bobdigital

Are you searching for expired domains that are linked to other websites?

Linked from, yes. For example, go through digg.com's archives. Find a site that got 5000 diggs but is now expired. That's the idea.

Sufyan · 03-31-2008, 10:08 AM

Quote:

Linked from, yes. For example, go through digg.com's archives. Find a site that got 5000 diggs but is now expired. That's the idea.

Did you find a single site that got 5000 diggs but is now expired? Curious.

RegFee · 03-31-2008, 10:33 AM

Originally Posted by seojunkie

Quote:

Linked from, yes. For example, go through digg.com's archives. Find a site that got 5000 diggs but is now expired. That's the idea.

Did you find a single site that got 5000 diggs but is now expired? Curious.

Not yet, only tried it with digg for a few seconds. But considering the amount of sites that are on digg, I'm sure it's possible to find one with at least a few hundred diggs. 5000 might be pushing it.

RegFee · 04-26-2008, 11:21 AM

Hopefully this will be available in about 2 weeks.

gazzip · 04-26-2008, 11:29 AM

Can you please send me some traffic to DomainFox.com

.

RegFee · 04-26-2008, 12:19 PM

Originally Posted by gazzip

Can you please send me some traffic to DomainFox.com

????: NamePros.com http://www.namepros.com/showthread.php?t=449536

Maybe if the extension becomes popular enough, you'll get some by default. But I like to think domainers are less prone to typing typos.

Are you developing a domaining extension as well?

squid · 04-26-2008, 01:14 PM

If it were possible, you could have something that turns the names green if they're unregistered - so if i were to be browsing digg and find an article with 1000 diggs and it turns out to be a typo, i could reg the name and get lots of traffic.

RegFee · 04-26-2008, 01:44 PM

Originally Posted by squid

turns the names green if they're unregistered -

Thanks for the suggestion, rep added.
????: NamePros.com http://www.namepros.com/showthread.php?t=449536

I really want to integrate availability checking into this extension, but will leave that for later since it will require an api access (javascript can't connect to a whois server) and I can't really afford to have people running thousands of queries right now. Hopefully Google can add its new DomainTools Whois Check into its API results, then that shouldn't be too hard to integrate.

Sufyan · 04-26-2008, 03:05 PM

Just so you know... I have a friend who has been tracking thousands of stories dugg for a long long time and stored them in a mysql database. I once asked him to extract domain names from the story URLs and check them for avail and to my surprise not a single domain was available out of around 250,000 URLs.

estibot.com · 04-27-2008, 05:10 AM

Sounds great Jorge, and I think there are several things you could do with this that would increase its usefulness. I think seojunkie is onto something, it's hard to find available domain names that have a lot of backlinks, very hard.

I am especially interested in the auto-surf function, it would be interesting to see what sort of turnout you get from having your bot surf for a day. If you can combine that to auto-checking for availability and maybe for overture/google, then have it list the results in a nice sorted manner, you would have a real nice tool that people might actually pay a little bit of money to use - I say a little bit because I now know that domainers are cheap

. With a little bit of revenue you could justify making even more improvements and perhaps even paid API access (Amazon has a reasonably priced API that will give you alexa rank, some whois data and traffic stats).
????: NamePros.com http://www.namepros.com/showthread.php?t=449536

DomainTools API is hugely expensive. I don't know how anyone can afford it. I've contacted them about special pricing for limited functionality, but they don't seem to answer emails.

Can you make this interact with the Firefox google toolbar in order to record the pagerank for each domain?

I would be interested to have the type-in recorder - you are right about the privacy issues, but there is still an opportunity to use it with the user's consent - - for instance, my wife lets me go through her Google search history so I can research what mothers of small children look for on the Web. I would love to install a type-in recorder so I could get data on possible available domains with type-in traffic.

RegFee · 05-28-2008, 07:05 PM

Originally Posted by Sufyan

Just so you know... I have a friend who has been tracking thousands of stories dugg for a long long time and stored them in a mysql database. I once asked him to extract domain names from the story URLs and check them for avail and to my surprise not a single domain was available out of around 250,000 URLs.

Sorry I didn't realize anyone else replied to this thread. Wonder why it didn't show up in my subscriptions. Oh well.

That is very very valuable information, thank you for sharing that. I actually went through quite a bit of digg pages and couldn't find anything available. Thanks for keeping me from searching them all!

Originally Posted by estibot.com

Sounds great Jorge...

Thanks Esa. Your estibot is an invaluable resource to domainers, I hope DomainNameFox can achieve its same level of usefulness.

Interacting with the Google Toolbar would probably be a good way of minimizing scan time. Thanks for the suggestion.
????: NamePros.com http://www.namepros.com/showthread.php?t=449536

I am hoping to explore the new domaintools/google whois feature. If google's API can provide just the simple results that a 'whois domain.com' search provides right now, that would be more than enough for my purpose.

TESTERS NEEDED

Really close to coming out with a finished product, so I need a few people to test the current version out before I release it.

Please PM me if you would like to help. Because I have yet to release this tool, I'm going to have to ask for well-established members only. I'm sure you understand.

Thank you very much!

gazzip · 05-29-2008, 11:21 AM

Originally Posted by RegFee

Maybe if the extension becomes popular enough, you'll get some by default. But I like to think domainers are less prone to typing typos.
????: NamePros.com http://www.namepros.com/showthread.php?t=449536

Are you developing a domaining extension as well?

True, I was going to be doing a domain search ap with a mate (the developer) but not sure if we have the time anymore.

Was that you that made a $70 offer for it at sedo yesterday ?

.

03-26-2008, 03:07 PM	THREAD STARTER #1 (permalink)
RegFee DNOA Certified Seller Join Date: Apr 2005 Location: Arizona Posts: 3,680	Need Suggestions - Finishing up making a domain name extracting firefox extension. I've created a link extracting/domain filtering firefox extension which is almost near completion. Here's a screenshot: So far it does the following: Every time the page changes, it scans for links (<a href="....">) and cleans the domain names out of the list. Then it removes all duplicate entries. Finally, if the list gets to a preset length, it basically cuts and pastes the list into a text file. Some cool things I've done with it: Combined with an auto-surfing bot extension, the tool harvests domain names from around the internet by searching google for a predetermined keyword, then following all of the links. Going through digg.com (or any site) archives and find available names Visiting link directories and just surfing around, collecting potentially available domain names (with at least one backlink, obviously) Just leaving it running while doing normal everyday surfing. Here's some things I've considered adding: Real time availability checking - not really practical since I'll have to get my own domaintools api license ($$$$) or require users to get their own. Option to send list to registrar for avail. check (ie Moniker bulk script) - got this working but it opens in the sidebar instead of the main window. Since this is my first major javascript project, I'm just going to leave this feature off for now rather than rack my brain. Option to change how many domains get harvested before sending to file. Option to not save domains to a file in the first place - necessary because javascript pops up the standard "are you sure you want to give this script access to save files?" warning and people probably won't feel comfortable with that. Do away with the sidebar altogether and make it run in the statusbar, saving domains as it goes along. The sidebar can't be closed while running a script, so it reduces the visibility. Ok, what I need help with: First off, I just need to know if there is any public interest in this. Secondly, if you are interested, which features would make it better for you. Sorry I don't have a beta version yet. I've got to clean the code up a bit. Thanks! -Jorge __________________ Nombrex.com

03-26-2008, 04:38 PM	#2 (permalink)
Spade NameBio.com Founder Join Date: Sep 2006 Location: Carlsbad, CA Posts: 7,855	Jorge, Sounds like a really cool project. Look forward to seeing what you come up with in the end. Justin __________________ Read my Blog! NameBio.com - Search The Domain Sales Index Buying LLLL.com's ending in "CA"

03-26-2008, 11:53 PM	#3 (permalink)
CharlieA NamePros Regular Join Date: Jan 2007 Posts: 376	Very nice. I hope you get this completed. __________________ Turn Your Parked Domains Into Stores! << High Paying Cost Per Click Make A Pay Per Click Store For Your Parked Domains! << Hassle Free and Customizable

03-27-2008, 01:14 AM	#6 (permalink)
xrvel i love automation Join Date: Nov 2007 Location: xrvel.com Posts: 1,620 Follow @xrvel	Nice. What about sorting the domains alphabetically. - Kurniawan. __________________ Xrvel \| Free Wordpress Plugins \| Desain \| Bodoh \| WP License

03-27-2008, 08:29 AM	#7 (permalink)
bobdigital NamePros Regular Join Date: Jan 2006 Posts: 915	Are you searching for expired domains that are linked to other websites? __________________ Top 3 lines of this signature ------------------> for sale --------------------------------------> Just PM me ---------------------------------> You don't have to be black to ChatBlack.com

03-27-2008, 12:18 AM	#4 (permalink)
netfleet NamePros Regular Join Date: Nov 2006 Location: Sydney, Australia Posts: 714	Sounds great! Would love to beta test! How about also harvests URLs that people type into their address bars? Obviously as domainers, we'd naturally check availability of any domain we happened to type in that didn't resolve but would be great to have something that automatically stores these type-ins for each computer. Then you'd just install it on each computer in your internet cafe & gain some really valuable info! Is this possible?

04-26-2008, 11:21 AM	THREAD STARTER #11 (permalink)
RegFee DNOA Certified Seller Join Date: Apr 2005 Location: Arizona Posts: 3,680	Hopefully this will be available in about 2 weeks. __________________ Nombrex.com

04-26-2008, 11:29 AM	#12 (permalink)
gazzip First Time Poster ! Join Date: Apr 2005 Location: GB Posts: 6,881	Can you please send me some traffic to DomainFox.com .

04-26-2008, 01:14 PM	#14 (permalink)
squid Account Suspended Join Date: Jan 2008 Location: Stafford Posts: 993	If it were possible, you could have something that turns the names green if they're unregistered - so if i were to be browsing digg and find an article with 1000 diggs and it turns out to be a typo, i could reg the name and get lots of traffic.

04-26-2008, 03:05 PM	#16 (permalink)
Sufyan NamePros Regular Join Date: Feb 2006 Location: Dubai, UAE Posts: 537	Just so you know... I have a friend who has been tracking thousands of stories dugg for a long long time and stored them in a mysql database. I once asked him to extract domain names from the story URLs and check them for avail and to my surprise not a single domain was available out of around 250,000 URLs.

04-27-2008, 05:10 AM	#17 (permalink)
estibot.com Developer of Estibot Join Date: Jun 2006 Posts: 1,945	Sounds great Jorge, and I think there are several things you could do with this that would increase its usefulness. I think seojunkie is onto something, it's hard to find available domain names that have a lot of backlinks, very hard. I am especially interested in the auto-surf function, it would be interesting to see what sort of turnout you get from having your bot surf for a day. If you can combine that to auto-checking for availability and maybe for overture/google, then have it list the results in a nice sorted manner, you would have a real nice tool that people might actually pay a little bit of money to use - I say a little bit because I now know that domainers are cheap . With a little bit of revenue you could justify making even more improvements and perhaps even paid API access (Amazon has a reasonably priced API that will give you alexa rank, some whois data and traffic stats). ????: NamePros.com http://www.namepros.com/showthread.php?t=449536 DomainTools API is hugely expensive. I don't know how anyone can afford it. I've contacted them about special pricing for limited functionality, but they don't seem to answer emails. Can you make this interact with the Firefox google toolbar in order to record the pagerank for each domain? I would be interested to have the type-in recorder - you are right about the privacy issues, but there is still an opportunity to use it with the user's consent - - for instance, my wife lets me go through her Google search history so I can research what mothers of small children look for on the Web. I would love to install a type-in recorder so I could get data on possible available domains with type-in traffic. __________________ FREE 40+ page eBook on Domain Development Download now! Start Your Own Domain Registration Business! Estibot Reseller Plans

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Need Suggestions - Finishing up making a domain name extracting firefox extension.

Testers Needed