DataBase Errors

[Eric Lyon]

Please bear with us, we have been getting hit hard with (possible) ddos attacks the last couple days on & off and still trying to block all the sources. Every time we seem to get it under control they switch to different ip ranges and start it again. It's very annoying I agree and we are working behind the scenes to try and eliminate this.

The first set of attacks appeared to be stemming from some of the chinese software bot spammers that were banned in the last 3 / 4 days. It was as if their software turned on at 10pm & ran till 1am pelting the servers trying to regain access automatically over & over sending hundreds of queries per second each.

Later, the incidents started happening at different times on & off with a few stable patches in between. So now we aren't sure if its chinese bot software's, a targeted malicious attack, or maybe something else.

Hopefully we'll be able to get all this resolved soon.

Sorry for the inconvenience and thanks for understanding.

Eric Lyon

 

[DN BROKER]

Thanks for the update Eric. I was wondering what the issue was for the last 12+ hours. I've issued an alert to the community on my blog and asked anyone with information to email me.

Cheers

 

[silentg]

Are they still DDOSing NP? I am still getting the database errors.

 

[RazorNF]

Its becoming quite aggrivating, especially while in the middle replying or posting, and hitting preview or submit to get one of those errors.

 

[Eric Lyon]

Are they still DDOSing NP? I am still getting the database errors.

Yes, We have narrowed it down to Bot attacks and are still working on a solution.

Its becoming quite aggrivating, especially while in the middle replying or posting, and hitting preview or submit to get one of those errors.

I agree, it's very annoying. Hopefully we can get this resolved sooner than later.

Sorry for the inconvenience,

Eric Lyon

 

[Bannen]

Good luck Eric

Wish there was a way to just shotgun all the ddos bots with one blast

 

[trainhappy]

I was wondering why I couldnt get to the site. Anyways, keep us updated.

 

[hollywood]

Wish there was a way to just shotgun all the ddos bots with one blast

Try rejecting at the router, not once they've reached the box? Good luck Eric - hollywood.

 

[Eric Lyon]

Try rejecting at the router, not once they've reached the box? Good luck Eric - hollywood.

We have plans later to put everything behind a new firewall which should rectify the situation completely. For now I'll try to keep manually heading them off at the pass to keep the server loads down enough that we have fewer outages (fingers crossed), well, unless I'm sleeping, then it might build back up again till I wake up (gets some fresh coffee ready).

I'll look into the router option a bit to see what's available.

Thanks,

Eric Lyon

 

[Archangel]

These attacks aren't new to NP. We've had them over the years. Here's to hoping for a smooth fix.

 

[clicktechs]

Sorry you guys are under attack. I certainly know you are trying to fix as hard as possible. Good luck or whatever the right saying would be on this.

Spam, bots and few other things can make the net a drag at times.

 

[Nathan]

We have plans later to put everything behind a new firewall which should rectify the situation completely. For now I'll try to keep manually heading them off at the pass to keep the server loads down enough that we have fewer outages (fingers crossed), well, unless I'm sleeping, then it might build back up again till I wake up (gets some fresh coffee ready).

I'll look into the router option a bit to see what's available.

Thanks,

Eric Lyon

Eric,

Another way would be to make a script to automatically ban an ip after x amount of queries per seconds. For example; the server would block the ips that hit 100 queries (depending on the excessive amount of queries) per second.

 

[Eric Lyon]

Eric,

Another way would be to make a script to automatically ban an ip after x amount of queries per seconds, that way the server would block the ips that hit 100 queries per second, or something along those lines

Another good suggestion to look into.

Thanks

 

[snowbird]

Means that these days database error may occur?Do not be surprised to encounter such a situation, is it?

It is not out of the ordinary for this to happen considering a forum like this that is database driven is using dynamic content that requires a database connection. Since dynamic content is more load intensive than loading static content, there are limitations placed on how many database connections there can be allowed at one time.

Ironically Eric, I saw you mention Chinese IPs and I recall about a couple weeks ago one of my servers being hit with failed brute force attempts on my ssh port from IPs in China. Apparently that is becoming a pretty popular location for obtaining a proxy.

As for DDOS, unfortunately it is an attack that is hard to prevent these days. There is no sure thing protection against it and the best way to mitigate them is to have lots and lots of bandwidth access to the backbone. DDOS is not really the problem here, the problem is all the botnets that are out there generating these attacks and with all the broadband internet connections made available to so many out there, it is becoming easier and easier to perform these kinds of attacks. Another method to help mitigate these problems is using Anycast which takes the server IP for namepros and routes it to several different servers hosted at several different hosts. In this method, people who hit the server's IP will be routed to the best available server based upon resource usage and location. This would help distribute the traffic through several networks vs. just 1.

Those who say to create a script to just ban the offending IPs, that can work for smaller DDOS attacks, but for serious attacks, the network is still being hit regardless. But that might be a good course of action at this time however since we are in fact seeing a database error which is a good sign. We are reaching the web server. There just appears to be too many database connections probably which is preventing some people from being able to access the dynamic content.

 

[Eric Lyon]

well said snowbird and thanks for the ideas. That basically summed it up. early today we opened up 200 more ports just to see what the bots would do & sure enough, regardless how many ports we opened, they filled them up within 60 seconds.

I've re-enabled a few features that were collecting dust over the years that query a central spam database multiple vb users contribute to and initiated the auto blocking sequence again for known spam sources. It seems to be trimming down lots of the manual workload and leaving me with a more manageable amount of bots that slip through the cracks (never reported before).

The downside to this is that occasionally there will be some legitimate IP's / emails / usernames that get caught in the net and denied access. For now though it seems to be a temporary solution to lighten the load. (At least till we get hit by the full force blasts again, which happened occasionally)

Once we are able to get a new firewall in place I think it should help resolve most the issues.

Eric Lyon

 

[snowbird]

well said snowbird and thanks for the ideas. That basically summed it up. early today we opened up 200 more ports just to see what the bots would do & sure enough, regardless how many ports we opened, they filled them up within 60 seconds.

I've re-enabled a few features that were collecting dust over the years that query a central spam database multiple vb users contribute to and initiated the auto blocking sequence again for known spam sources. It seems to be trimming down lots of the manual workload and leaving me with a more manageable amount of bots that slip through the cracks (never reported before).

The downside to this is that occasionally there will be some legitimate IP's / emails / usernames that get caught in the net and denied access. For now though it seems to be a temporary solution to lighten the load. (At least till we get hit by the full force blasts again, which happened occasionally)

Once we are able to get a new firewall in place I think it should help resolve most the issues.

Eric Lyon

Thanks for the update. I hear Cisco Guard is a pretty good firewall that also helps assist in managing and reducing DDOS attacks.

 

[earthgld]

Until i post this, still i often get the database error page. So the DDOS is the cause of all this, hope the team can get rid off the attackers. Good luck.

 

[Arief WN]

... For now I'll try to keep manually heading them off at the pass to keep the server loads down enough that we have fewer outages (fingers crossed), well, unless I'm sleeping, then it might build back up again till I wake up (gets some fresh coffee ready) ...

Eric, I think you need some good wingmen to back you up...

 

[GiftedDomains]

Those jobless scumbags doin this should be put in jail.

 

[Kamikaze]

Good luck, Eric!