Dynadot

FREE [FREE] Useful Python Script to filter expiring domain lists by customizable criteria

Spaceship Spaceship
Watch

Croc

Upgraded Member
Impact
322
I wrote a little Python script this week to aid myself in filtering daily expiring domain lists by criteria I specify. I have it set to run automatically using the built-in Automator application on macOS, but it can easily be run with a UNIX cronjob or Windows Task Scheduler as well.

You can review the full source code at github.com/crock/dropcatch-filter
If you are knowledgeable in Python, contributions are welcome. Just make a pull request anytime and I'll review it as soon as possible.

Comes with a simple JSON configuration file that looks like this:
Code:
{
  "maxDomainLength": 10,
  "keywords": [
    "minecraft",
    "coin",
    "craft",
    "tech",
    "pvp"
  ],
  "tlds": [
    "com",
    "net",
    "org",
    "io",
    "co"
  ]
}

The script programmatically downloads the next day's expiring list from Namejet's website and parses it. It then filters the entire list in mere seconds based on the criteria you specify in the config.json file.

If running on a timer of some sort, please set the script to run AFTER 7:00 AM PST because the next day's list is not available until that time the day before.

Installation Instructions

  1. Download the Python 3.6.x+ installer from python.org/downloads
  2. Run the installer, but please make sure to tick the box labeled "Add Python to environment variables". It is unchecked by default. This is important!
  3. Open Terminal (Mac/Linux) or Command Prompt (Windows) and type the following command to let the terminal know where the script files are located. Replace [file path] with the full path to the directory containing the files: cd [file path]
  4. Next, run the following command: pip install -r requirements.txt
  5. The above steps only need to be run once. Now every time you want to run this utility, you simply run the following command: python main.py (if that doesn't work, try replacing python in that command with python3)
  6. If executed correctly, you will see domains being output according to your criteria and they will also be saved to a file in the results directory with the name results_MM-DD-YYYY.txt



 
16
•••
The views expressed on this page by users and staff are their own, not those of NamePros.
Cool I’ll check this out today.
 
2
•••
Thanks to a generous soul from another forum for making a pull request. With the latest change, Python 3.6.x is now the base required version going forward.

If you need a backported version (3.x.x only) for some reason, please send me a message.
 
1
•••
Just pushed a critical fix for a bug on Windows systems that prevented the script from running properly. Go download the latest zip from Github when you get the chance.

Thanks to a Discord user for reporting this bug!
 
1
•••
Nice.......i'll check it out
 
1
•••
Can you also add configs to search for alphabetical domains only (i.e. exclude hyphens and alphanumeric domains)?

Thanks
 
0
•••
Can you also add configs to search for alphabetical domains only (i.e. exclude hyphens and alphanumeric domains)?

Thanks
For now, I've edited the Regex in Filters.py

Thanks
 
0
•••
I built a new config generator for those non-developers that want to use my tool. Also, I have ported this application to an exclusive Discord bot command for my domain name community.
https:// rule_violation [dot] com /dropfilter/config/
 
0
•••
thanks, will check it out soon
 
1
•••
@CrocBuzz Getting this error - Couldn't get file.
 
0
•••
0
•••
Here is the entire error log from console.
Downloading tomorrow's list of expiring domains...
Couldn't get file.
Filtering domains according to your specified conditions...
Traceback (most recent call last):
File "main.py", line 68, in <module>
main()
File "main.py", line 63, in main
domains = [line.rstrip('\n') for line in open(os.path.join('tmp', filename))]
IOError: [Errno 2] No such file or directory: u'tmp/2-12-2020.txt'
 
0
•••
Trying this on a remote shared server on linux
python 2.7.5
pip 20.0.2
 
0
•••
Here is the entire error log from console.
Ah I see, is your timezone somewhere around Australia because that's a day ahead and Namejet's list is only available at 7am PST. I've fixed this issue using the exclusive Discord bot version of my program, but you would have to join my community in order to use it. Here's an invite https://domain-api.now.sh/api/discord/invite
 
0
•••
Unfortunately my host admin says that they dont "officially" support python apps. So they wont be upgrading to Python 3.x.x. Before I give up, I want to give this one last try, if it can it work in Python 2.7 with some code modifications or installing dependencies etc.

Do we need to convert the american timezone to our local.
Code:
local = arrow.now("America/New_York")
tomorrow = local.shift(days=1).format("M-DD-YYYY")

I can get this done in javascript or php but Python is said to be second just to Perl when it comes to text parsing. Wish I knew Perl. So it is a performance thing. Beause I do not know yet how big the data will be in the future, so I wanna start on the right foot.
 
Last edited:
0
•••
Unfortunately my host admin says that they dont "officially" support python apps. So they wont be upgrading to Python 3.x.x. Before I give up, I want to give this one last try, if it can it work in Python 2.7 with some code modifications or installing dependencies etc.

Do we need to convert the american timezone to our local.
Code:
local = arrow.now("America/New_York")
tomorrow = local.shift(days=1).format("M-DD-YYYY")

I can get this done in javascript or php but Python is said to be second just to Perl when it comes to text parsing. Wish I knew Perl. So it is a performance thing. Beause I do not know yet how big the data will be in the future, so I wanna start on the right foot.

The Python cli is behind on updates. That's why it's free and open-source. The bot command version is a Node.js/Javascript port with more functionality, but I haven't open-sourced it yet. You can interact with the bot command on my Discord server.
 
0
•••
@Crock
Currently trying this on a remote shared server on linux
python 3.6.0
pip 21.1.1

I am getting following errors when I run the script with python3 main.py

[username@lu-shared02 pyapp]$ python3 main.py
Downloading tomorrow's list of expiring domains...
Traceback (most recent call last):
File "main.py", line 72, in <module>
main()
File "main.py", line 64, in main
download_file(url, filename)
File "main.py", line 28, in download_file
file_size = int(response.headers['Content-Length'])
File "/home/torje/pybase/lib/python3.6/site-packages/requests/structures.py", line 52, in __getitem__
return self._store[key.lower()][1]
KeyError: 'content-length'
 
0
•••
@Crock
Currently trying this on a remote shared server on linux
python 3.6.0
pip 21.1.1

I am getting following errors when I run the script with python3 main.py

[username@lu-shared02 pyapp]$ python3 main.py
Downloading tomorrow's list of expiring domains...
Traceback (most recent call last):
File "main.py", line 72, in <module>
main()
File "main.py", line 64, in main
download_file(url, filename)
File "main.py", line 28, in download_file
file_size = int(response.headers['Content-Length'])
File "/home/torje/pybase/lib/python3.6/site-packages/requests/structures.py", line 52, in __getitem__
return self._store[key.lower()][1]
KeyError: 'content-length'
I don’t really support this script anymore. Use the web version instead as it is better and has more functionality at https://dropfilter.app
 
Last edited:
0
•••
1
•••
Back