Jason, I reprinted a big chunk of your article here

http://capitalregionpeople.blogspot.com/2006/05/take-that-wemedia.html

(and linked to your pages from said chunk)—- I am wondering how YOU feel about having a back-up or alternate blog on a totally different service. There have been times in the past when blogspot has done “housecleaning” and deleted good blogs that they “thought” were spam or had been abandoned. Have you had many outages with your blog service provider? Inquiring minds want to know ;)

BWAHAHAHAHAHA! I do so hope they get totally taken to the cleaners. I hope you don’t mind that I’m passing this one along to my flist. As they say, this is so going in my livejournal.

Oh, and I did briefly try that frog. Canned it as it kept crashing Firefox just about every time I opened my gmail. That and I didn’t notice any reduction in spam.

• Posted by: Rebekah on May 3, 2006, 8:37 PM

Personally, if I’m hosting content that I have a feeling is going to be a huge blockbuster for some reason, and generates thousands or millions of hits, I’d post to a service — like LiveJournal — that I don’t have to admin or maintain. I pay for the service, and there’s no limitations in the contract they place me under that say “You can not redirect traffic our way under ondition of $foo”.

I would expect that BlueSecurity’s thoughts were similar: “Our (Network|Hardware|Setup) can’t maintain this traffic level, but we pay for hosting through another service that can. Clearly, we should use that service, for which we’re paying , to its fullest.”

Additionally, simply changing where the domain pointed wasn’t all of the problem: The DNS servers, based on my experience, seemed to be under a malicious attack, since they weren’t responding to pings at all. I could be wrong, and that could simply be a failure due to excessive NS lookups (I guess) but it seems unlikely: The servers *were* attacked, regardless of the reason for it, and the fact that BlueSecurity was involved does not change the fact that the abusive behavior was directed towards SixApart.

Additionally, LiveJournal at least used to have portions in their ToS regarding identifying users. Identifying troublemakers on your network, especially when they are commercial in nature, is a great way of asking for a lawsuit, so I certainly don’t blame SixApart for being vague.

Lastly, the biggest thing that DDoS type people want to see is their name in lights. Posting about “Spammers attacking bluesecurity” is only likely to make them redouble their efforts. In the past, most communication about DDoS situations with LJ have been kept relatively quiet so that people don’t feel like they’re achieving their goal. Obviously this is not nearly as possible when you’re down for four hours, but not putting the BlueSecurity name into the status announcements just makes sense.

• Posted by: Christopher Schmidt on May 3, 2006, 9:58 PM

Wow. Thanks for this.

• Posted by: Prissi on May 3, 2006, 10:39 PM

Note that after reading a couple more links, it does seem likely that BlueSecurity did this knowing that they were under a DDoS attack, and knowing that the attack would be redirected, even if they weren’t exactly sure how: in that case, it seems that they would be in violation of the Terms of Service which prevents users from intentionally impairing the usability of the service to other users.

However, I still maintain that such a decision could in some cases be made in ignorance of the possible damage it could cause. I don’t know enough about the situation to say whether that was the case here. I just like to play devil’s advocate, for the most part.

• Posted by: Christopher Schmidt on May 3, 2006, 11:06 PM

Shameless, although not surprising.
I hope to see them go down hard for it.

What the shit. Saying that I hope they are sued is not what I really wanted to say, but since this is a public place I’ll keep what I really want to say to myself. Let’s just say it involves rusty scissors.

something I’m pretty damn sure isn’t part of the TypePad service agreement.

Since when does Typepad care about blogs they host following the agreement? Basically, you have to create a PR nightmare for them to get them to dump the offenders.

Dugg this.

Blue Security are fine!
They did NOT do anything bad. They are fighting for us the battle against the spammers who currently rule the Internet.
They have a great service for protecting email addresses from spam. They even had a big success recently: many spammers stopped sending spam to the Blue community. My spam reduced in more than 50%.
Apparently, one spammer got mad and started attacking all the systems of Blue Security, taking down any ISP in the way.
TypePad and LiveJournal were attacked too.
You should blame the spammers that attacked. Not Blue Security.
We should find the spammers and get the control on the Internet back to our hands!

Everyone should join BlueSecurity right now. This is a war and Blue Security is in the front. Read this before you judge them:
http://www.comagz.com/webmagazine/nir/support_bluesecurity_now_its_your_chance_to_fi

wow, i wondered why about four of my regularly visited blogs where down dismorning…

Incredible.

Ah, I see the BlueSecurity sock puppets have appeared. Been seeing a lot of those in comment sections recently ;)

• Posted by: jm on May 4, 2006, 4:59 AM

Oh boy do you show your ignorance of this affair.

I understand your nose is out of joint a little, but your rhetoric is way off the mark.

Bluefrog is not a DDOS service, or it would have folded long ago.

Instead of blaming Blue Security for your outage, you should be pointing at the spam crews who are actually performing this action.

jm,

I think Blue Security guys are pretty busy today.

However, if you have any questions about how bluefrog works, or what it does, I’ll stick around a bit and try to help.

I am a bluefrog member, not staff, but I’ve been with BF (or BS) for about 6 months, so I’m pretty familiar with the program.

First of all, I agree with most posts here that there is no way that Blue Security would intentionally do this.
I can’t understand why their method is “contreversial”. For every spam you get, the spammer gets a “remove me” email back at them. It’s not an attack, it’s 100% reasonable and 100% legit. When you get an email from a person, they can’t argue that you should not answer it, just because it’s spam.

Sadly enough, the Blue Frog software doesn’t work with POP3 email, only web services, so I can’t use it - but when there’s a version for outlook or thunderbird, I’ll definetly install it.

Neko,

BlueFrog doesn’t send any emails. The user agent occasionally posts opt-out requests at websites that are selling the things that you have been spammed for.

These opt-outs are not simultaneous, and not high-volume. If you get 1 spam, you *might* send one opt-out. Usually a few days later on.

Nothing goes to the sender of the email (or their IP address.) If the originator of the email (the spammer) is identifiable, BS approach them and ask them to wash their mailing list of BF subscribers.

BF works fine with my pop3 account (though it is not completely automatic.)

Oh, that’s all a little harsh isn’t it - wanting BlueSecurity to fold?

Yes, i’m a BlueSecurity user, and I like the idea of their project.

And yes, I agree changing their DNS records to TypePad possibly wasn’t the best idea - but they (BlueSecurity) wanted a way to be able to communicate with their users over what was happening, a blog was a logical choice.

During the first 24 hrs of the attack, when BlueSecurity’s website didn’t work at all, I very much wanted some kind of update on the situation, and so I was quite happy to see the Blog be put up.

As I don’t work with BlueSecurity, or have any contact with them other then being a user, I can’t speculate on their reasons for changing their DNS records to TypePad, perhaps they thought since TypePad is considerably larger, it would handle the DDoS attack without any problems? Perhaps they thought the spammers would stop attacking once BlueSecurity was down (which it is). Perhaps they thought the spammers would continue to hit BlueSecurity IPs, rather then move over to TypePad?

Who knows. But I respect BlueSecurity as a company, and I would like to think they didn’t have any kind of evil intentions by changing their DNS records.

I’m also a regular user of LiveJournal (ie. thus TypePad/Six Apart), and I noticed LiveJournal’s post about a DDoS attack, without putting the two events together, assuming they were separate attacks.

BlueSecurity is here to help us all, and, it’s obviously making an impact on the spamming community if they are going to these lengths to bring it down.

So, to review: Has Blue Security done the wrong thing? Maybe. Do they deserve to be “taken to the cleaners for this one” - no way.

• Posted by: DWZ on May 4, 2006, 5:34 AM

Perhaps These great champions Blue Security should fight their own battles, instead of passing the buck

marc,

I agree. I’ve had my domain knocked off the air today too (probably a knock-on of the ddos, I’m with domaindirect) and it is a bit annoying.

However, I think this sort of thing can make people start to realise that passively giving in to spam is not a solution either. If they are happy to cause this sort of easy chaos, why would you tolerate them?

Most DDoSes I’ve seen are IP-based, not domain based. Most botnet sources I’ve looked at don’t exactly support packeting by domain

Well, if a DDoS happens in Israel, Americans look at it and say “Sucks to be you.” Blue Security needed law enforcement help to go after the spammers who seem to be controlling one of the world’s largest botnets, and frankly, Israeli law enforcement may not be up to the task. What is a small Israeli company to do?

They redirected their DNS services to their blog site that was hosted at Typead. The result is that the DDoS chased them there, and it crushed Typead’s hosting network, Six Apart. Live Journal is also part of Six Apart, so it went bye-bye too.

Lots of people on the net are figuring out what happened now and many are understandably pissed, but they are directing their anger at the wrong people. They should be mad at the spammers who are criminals and engaged in a criminal act. All that Blue Security did was make a DNS change. Which is legal.

Sure, they probably knew what would happen. And they also knew that if the American critical infrastructure was attacked by the world’s largest spammers with the world’s largest botnets, then the Department of Homeland Security, the FBI and the Secret Service would bring their considerable law enforcement skills into the fray, and help track down these criminals and put them away.

I agree with sonth, it sounds really weird that they would use the domain for their DDoS and not the IP itself. But even if they did and the attack was redirected to take down another site, I don’t think this was intentional as Blue Security probably thought the attack would still be directed at them, but they would have a page up and running.

Amusingly, bluesecurity.com is now pointed at 127.0.0.1 which is what they could have done in the first place rather than being a rather ungracious netizen. I would hope that 6A would sue them for the additional unsolicited traffic (wait, isn’t that what spamming is in the first place?!) for the cost of the bandwidth. There is simply no excuse for throwing your garbage on someone else’s lawn, and no excuse that bluesecurity or their users can say to justify this action.

-Any- retaliatory attacks are not effective on the Internet, period. All it does is strain the infrastructure further until the attack/counterattack is eliminated from a higher authority, blacklisting IP blocks and eventually entire subnets.

Regardless, bluesecurity violated their TOS with their uplink, which -undoubtedly- states that you can not use their service to attack, DDoS, or otherwise assault other servers/sites.

—Cryo

• Posted by: Cryo on May 4, 2006, 8:12 AM

I think Blue Security did precisely the right thing. Apparently, once its site went down, Blue Security decided that it owed an explanation to its members and set up a blog. If the spammers weren’t so malicious, at that point they would have called off the attack. Instead, they continued the attack, following the redirect to typepad.

So apparently the spammers not only want to take down Blue Security, but also prevent it from being able to communicate with the outside world at all. Moreover, it seems they don’t care how many other people they effect in the course of doing so.

Put another way, if Blue Security now sets up a simple blogger account, doesn’t point bluesecurity.com at it, but relies on ‘word-of-mouth’ for people to find it, we can assume the botnet will go after that too. Should Blue Security just sit quietly and shut up?

I would have thought that the criminals running the botnet are 100% to blame. Blue Security operate the *only* even partially effective anti-spam solution on the net. I hope they find a creative anti-bot solution fast.

To those defending Blue Security’s spam-fighting methods as anything other than a denial-of-service in and of themselves, remember that that’s the whole point of Blue Security’s methods, to shut the spammers down via an onslaught of traffic. The folks at Blue Security aren’t trying to help everyone unsubscribe from spam, they’re trying to overwhelm the spammers — which is, by definition, a denial-of-service attack.

From the PC World article quoted in the original post:

Blue Security follows the links inside the body of the spam message… [and] then identifies the form fields at the spammer’s site (where you’re asked to input credit card data, for example) and then uses the software you installed to direct your PC to insert in those fields a request to unsubscribe you from the site’s mailing list…. Blue Frog’s software causes all of its connected users to submit the request/complaint simultaneously—and repeatedly—for a period of time.

From Wired:

Here’s how the technique works: When users add e-mail addresses to a “do-not-spam” list, Blue Security creates additional addresses, known as honeypots, designed to do nothing but attract spam. If a honeypot receives spam, Blue Security tries to warn the spammer. Then it triggers the Blue Frog software on a user’s computer to send a complaint automatically. Thousands complaining at once will knock out a website and thus encourage spammers to stop sending e-mail to the “do-not-spam” list.

I am a Bluefrog user, and I have to admit, it was working. Took a little while, but my spam (except my yahoo.mail account) was seriously diminished (to the tune of 1-2 a day instead of almost 50, which was keeping Norton busy). I also appear to be one of the ones under spam attack now.

The thing is, I look at spammers like this the same as terrorists. They don’t like what we’re doing, so they’re going to try everything, including illegal means, to stop it. If BF was illegal, Interpol would have shut them down by now, I would assume. I know when they figure out where this DDoS is coming from they’ll be paying this SOB a visit. To me, that’s the difference.

Is it ‘fair’ the way BF works? Maybe, maybe not. But it’s not fair to our mail servers, ISPs, etc to receive 2-300 emails a week for viagra, breast enlargements, etc, when the e-mail is someone who doesn’t need it, or, as in my case, a child’s. That’s what pushed me over the edge. Whem my child’s new e-mail address (not a month old at that time) was already getting porn e-mails.

So you’ll forgive me if I don’t pitty the poor spammers that get spammed. Do unto others as you would have them do onto you.

• Posted by: WraithFive on May 4, 2006, 8:53 AM

Since when has requesting to opt-out become a criminal offence? That’s the ONLY thing Blue Security does. If it results in a DDOS phenomenon towards the spammers’ websites, it’s only because so many people have been mass mailed by the spammers themselves.

Don’t confuse it with a DOS attack. We’re only requesting opt out. Yes, the spammers’ servers are overloaded with traffic, but isn’t it what they wanted in the first place?

If they only want 1% of their recipients to log in, isn’t that fraud and discrimination? (Oh, you want to buy things? Welcome! You want to opt out? ACK! You’re EVIL!)

Well, at least the spammers are fighting, which means they are on the run!

Robin

• Posted by: robin.myopenid.com on May 4, 2006, 9:04 AM

Jason,

You have selectively quoted poor articles that have as little understanding of the bluefrog method as you do.

Bluefrog does not incapacitate web sites. That is not the point. It never was, and endlessly repeating that mantra will still not make it true.

Bluefrog is not DDOS. It does not deny service to anyone. For non-compliant spammers, it simply makes life uncomfortable for their customers (the people who are actually selling the viagra, pron, rolexes etc) by sending them a lot of bogus orders.

Bluefrog is simply a do-not-spam registry, with a mechanism for complaining about repeat offenders.

For every article you can quote saying that bluefrog is evil, there are three more that say it is a good system. This DDOS and the traffic on spammer newsgroups that I’ve read, simply demonstrates that the spammers are running scared of bluefrog.

One (spammer group) post I read indicated that the spammers truly feared the capabilities of a bluefrog client installed on 2 million computers and have initiated this attack to kill the system before it gets that big. Blue was already running for 500,000 in less than a year.

Now I understand that typepad users are upset about the DDOS, but pointing the finger at Blue Security seems to be missing the point.

Once the dust settles, I suggest you read a little further.

I’ll say it again, I understand that typepad users and others affected by this DDOS are upset (my mail server is still offline, and has been for the last 21 hours.)

But you have to understand that we’ve been granting the privilege of free reign to these guys and just because we (BF users) feel that apathy is not an option, these cyber-terrorists think they can run rough-shod over everyone.

Jason,

PC World is wrong on a couple of points.

1) Only users who receive a spam from a site are triggered to submit the opt-outs on that site. Not all the users.
2) This is not simultaneous. It is staggered. The point is not to kill the site.
3) It is not repeated. A maximum of one opt-out is submitted for each spam mail received by the client.

The folks going on and on about whether Blue Security is a good product or not are completely missing the point. The question on the table here is not whether spamming is good or bad. Nor is it whether Blue Security’s methods are morally appropriate or even technologically effective.

Blue Security set themselves up as a target in a cutthroat world - the spam world. They put themselves up as the defenders of people’s inbox, so naturally, eventually they would be attacked. That is the nature of the Internet. If you jump up and down and say “Look at me, you losers, I’ll shut you down! Bring it on!” - they’re going to do that.

But instead of accepting this challenge, they instead shunted this DDOS attack on their servers to the SixApart server clusters by redirecting the traffic away from their systems. There are many reasons why this proves how poorly run a company Blues Security is:

1) They did not inform SixApart of what they were doing. Yes, they simply made a DNS change, which is ‘legal’, but so is just turning on a water hose (using the metaphor someone else used earlier in this thread). The act itself was not illegal per se, but the intended result was.

2) They deliberately redirected an attack against them to someone else, thereby saving their own skins at the expense of someone else - a company that has traditionally been very good to the ‘net community at large.

3) Redirecting this discussion from “Was their action justified” to “is their product worthwhile” is like arguing that Microsoft attempting to shut down a competitor is okay because they make a helluva good flight simulator. They just don’t relate.

Stay focused on the real issue at hand. Blue Security has shown an extreme lack of judgement and misunderstanding in how the net community operates. People should consider this before endorsing their products.

• Posted by: dbs on May 4, 2006, 10:42 AM

Dylan, I think you’ve made your point, but we’ll have to agree to disagree on this one. I haven’t seen one whit of documentation to support the claims contrary to the press coverage of Blue Frog (e.g., your assertions in your 9:19AM comment); there are enough reputable and legitimately-concerned people questioning Blue Security’s tactics with Blue Frog to convince me to be wary. And I haven’t even begun to address the other concerns I have with their methods, concerns that make their “solution” just as untenable as all the others. (As just one example, how would Blue Frog deal with a spammer sending out an ad that had twenty links in it, ten of which were to legitimate and unrelated sites like CNN, Blogger, and Google Groups? If the Blue Frog user apps then inundated these sites with “bogus orders” and nonsensical form submits, and caused the sites to slow down or cease operation as a result, would Blue Security take responsibility?)

Finally, and most importantly: regardless of whether Blue Security’s methods work, are kosher, whatever, the company specifically opted to take the hostname record for its main web presence — the hostname which was under a DDoS attack — and redirect it to someone else’s network. I now have it on pretty reliable background that no notice was given of this (of course, this doesn’t really matter either). And as I said above, yes, ultimately the spammers launching the DDoS are bad people, and are the ones who bear fundamental blame for all network outages related to their actions — but it’s hard to ignore Blue Security’s role in what ended up happening. You can take your burning sofa and throw it out of your house to prevent the whole place from catching fire… but you still bear some responsiblity when the sofa lands in the guy’s house next door, and his home burns to the ground.

With apologies to all the BF users, I still think Blue Security was in the wrong here. It seems that they simply referred the attack to someone else (‘dumped their garbage on someone else’s lawn’ was a good analogy). Whether they knew that this would happen (in which case they are doubly culpable) or not (in which case they are fools, for they knew or should have known), it’s what happened; saying ‘they provide a good service, at least they’re doing something about spam, blah, blah’ does not exculpate them from either evil intent or stupidity. They lose, as do we all.

Lets clear up a few points:
Spamming is illegal in most countries.
Launching Distributed Denial of Service attacks is illegal in every country.
Operating a botnet is illegal.
Compromising systems around the world in order to install and operate botnet software is illegal.
Owning a botnet that is large enough to crush one of the world’s most reliable ISP’s for blogging?
Yep, its illegal. It should also point out to you naysayers exactly who we are talking about here.

The spammers have a botnet than can crush google.
they also have access to the United States Critical core routers because they were able to alter BGP peering points in a way to blackhole Blue Security to be accessible to just Israeli addresses.
We are talking about a criminal enterprise here folks, not some whiny activities by script kiddies.

Also: Creating a list to opt out of spam- That’s legal.
Following the law and rules of the USA Can Spam act by sending requests to not spam? Yes, that is legal too.
Changing your DNS entry? Guess what? Its legal.

Changing your DNS entry in the hopes that it will attract the attention of DHS, FBI and USSS to the threat that this criminal enterprise poses to Ecommerce? That’s priceless. And… Legal.

Pat (BelchSpeak), I don’t even know where to begin with that one.

First, stop ignoring the repeated acknowledgement that the spammers’ own DDoS was a bad thing. Nobody’s debating that point with you; I think I can assume that everyone agrees that the people responsible for the original DDoS against Blue Security are scumbags.

Second, as far as I can tell, nobody knows enough about the origins of that original DDoS to make as definitive statements as you’re making (“compromising systems”, etc.).

Third, nobody’s debating your point about Blue Security’s opt-out list. Simple opt-out lists are fine.

Fourth, I haven’t seen a shred of evidence to support the contention that the originators of the DDoS had access to BGP tables within the US critical core. This Slashdot journal posting seems to be the nidus of this idea; it hasn’t been sourced beyond that, and doesn’t seem to have a source. (I’m prone to think that any routing changes which occurred during the DDoS were those made to protect the networks on which the target systems resided — changes that are supposed to happen as network operators trace DoS attacks and modify traffic rules to shelter the network from the storm.)

Fifth, people on both sides of the spam issue will debate you on Blue Security’s method of sending requests to opt out of spam; if you really believe that it’s as cut-and-dry as you represent it to be, then so be it, but it doesn’t appear to be.

Sixth, it’s insane to state in a vacuum that changing your DNS entries is “legal.” (“Firing a gun? Legal.” “Putting rat poison into a container? Legal.”) Nevermind how many issues that ignores, it’s just ridiculous to think that you have some intrinsic right to take an action that brings down another network.

And finally, the idea that it’s somehow OK to take an illegal action outside of the U.S. and redirect it within so as to “attract the attention of DHS, FBI, and USSS” is absurdly indefensible. Again, carry that through to its logical conclusion: if I discover a bomb outside an Israeli disco, is it OK to load it onto a plane and bring it to the foot of a U.S. monument so that I can garner the attention of the U.S. authorities? Absurd.

Any marketers who use unethical bulk-email services are just as responsible IMO. And if they don’t like the complaints F-em.

All the conjecture from reading a headline does not inform the public of what actually transpired regarding the DOS attack, or the actual operation of Blue Frog.

But to all of the eMail users who want the UCEs Don’t use the Frog! Or Waste your money on your filters and services so the spamvertisers can operate in ‘peace’ by hijacking your PCs. But I for one would rather get some of my bandwith back on my broadband by reducing some of the mail sent by your zombies.

My Frog will continue to croak, and loudly.

Interesting update as of 10AM on may 4th

Apparently the hackers got into the whois database and changed the ipaddress for bluesecurity to localhost…hee hee..

Oh well

Ummm, Markb, no — WHOIS has nothing to do with hostname DNS address mapping, and that change was likely made by Blue Security and/or their ISP so that the DDoS attackers would end up attacking their own machines.

Let’s try to make this thread even more productive.

If we have a little faith in the system, we can assume that Blue Security will be held accountable if it’s determined they did anything wrong, knowingly or not.

Let’s also not waste any more time debating the merits or sins of Blue Security’s tactics for dealing with spam. That’s a debate which has been going for months and will not be solved here.

Instead, isn’t there a higher value in looking at what has been revealed by this attack? BelchSpeak may have overstepped reality a tad by drawing some of his conclusions, but many of the observations remain valid.

There is clearly a criminal element that maintains sufficient control on the internet to wreak havoc with the economy and more. This is clearly a threat to national security and should be the most important topic raised by this issue.

Let the law enforcement and judicial system sort out who’s to blame and who must pay for this particular episode. But will we ignore the important writing on the wall that this episode has revealed?

I hope not.

It is very possible, and plausible, that even WITH knowledge the DDOS attack would follow the change of IP address, that blue had no idea that it would impact a high speed hosting provider. Blue is a small startup, I am sure with limited bandwidth compared to large US based hosting providers. I am sure they could not judge from their perspective how large the attack was. I dont feel it reprehensible to try and get a message out to their users by shifting their IP address to that of their blog. The VP of Six Apart is quoted here: “Blue Security is a customer of ours, they do have a blog with us,” Six Apart Vice President Anil Dash said. “Beyond that, I don’t want to confirm anything. Any kind of an attack like this is really the fault of the attackers.”

How COULD blue know that six apart would not be up to the volume? People bring traffic along with many posts on their blogs. They were a client, and simply used their page. Once they realized the damage, they changed the IP address to localhost.

To sit and judge from the safety of your desk job, the actions of a company that is literally in a fight for their existence, and then say they are at fault, is absurd! The blue founders have long and legitimate security backgrounds.

And once and for all, it is NOT a denial of service that causes these spammers pain from Blue, it is fake submissions through their websites that cause the problem. If a spammer has to sift through thousands of fake orders to find the 3 or 4 that are real, the business is not nearly as profitable!

And as for belchspeaks comments on BGP - here is a clip from an article on CBR where the founder of blue discussed the attack:

*****
After that, the attacker went after bluesecurity.com, somehow managing to have it rendered inaccessible to users in the US and Europe, while leaving it accessible in Israel, according to Reshef.

Reshef said the company has been in contact with the spammer via ICQ and that the spammer had claimed that he had carried out what he called a “backbone subversion” attack against a tier-one IP backbone.

Reshef added that the spammer had provided what purported to be a partial transcript of an ICQ chat between himself and an engineer at the backbone provider in question, in which the engineer agreed to be complicit in disconnecting Blue.

We could find no person or reference to explain whether such a thing as “backbone subversion” even exists, and spokespeople for the carrier in question had no information on the matter, so we won’t name the company here.

Reshef said he did not necessarily believe the spammer’s claims about the backbone provider and its engineer.
*****

I am sure if this turns out to be true, it would in fact be a BGP entry that was the culprit. Regardless of the method, if a tier 1 ISP was involved, then this criminal organization has far more reaching contacts than we thought possible.

Tim, nearly every point you raise has been addressed elsewhere in this thread, so I’ll leave it to you and others to match up the arguments.

That said, you haven’t a clue what background I have (“desk job”??), and what knowledge I bring to this situation, so I find your admonishment more funny than valid. (That, and your knowledge of what defines a “denial of service.” Your “thousands of fake orders” paragraph is particularly priceless.) And finally, I love that you “are sure” that a BGP entry will turn out to be the “culprit,” and use as verification a secondhand account of a thirdhand instant-messaging conversation about a network attack that nobody’s ever heard of. That’s the icing on the cake.

Hi, Jason, I’ve got a humble question for you. I’ve been reading the tone of the exchanges on this page and it’s clear that you are an intelligent person. So, can you explain to me, also a fairly intelligent person, what exactly your gripe is with Blue Security? You see, I keep encountering a lot of extremely hostile blogs aimed at Blue. Some of them are clearly run by spammers hoping to discredit Blue, which speaks plenty about Blue’s efficacy. Others are members of the online security industry who keep harping on about the DDOS phenomenon. These people I suspect break into at least 2 groups. There are those who genuinely have an ethical issue with Blue’s methods. These people I respect, though I disagree with them - in the same way that one respects Anti-War Buddhist monks for setting light to themselves. Ineffectual, but solid in their views. Then there are those who are working within a defeatist mindset who are unaware of the current culture within which they operate. Like the fish who is unaware of the water it swims in, they move in psychological environment which states: this is how it is and this is how it always will be. They equate, essentially to losers. There is also a third category: those who want Blue’s ideas to fail because they didn’t think of it first. There are plenty of programmers out there writing spam filtering apps who are very happy to see another harvest of spam emails coming through the system. It justifies their existence and gives them work. There has also been an interesting Anti-Jewish strain to some of the emails from spammers, which is utterly ridiculous. Do you fit into any of these categories? Or can you actually clearly state why you are so down on Blue Security? - Apart from: What they are doing is illegal - which is clearly from the amount of debate on the matter - debatable. Obviously, each one of the mindsets listed above affects the angle from which you criticise Blue. At the moment, I find your criticism difficult to pin down. Please, let me know the philosophical underpinning of your anti-Blue stance. I find it really fascinating that Blue’s methods should cause such a stir in the Net world, and I think I often encounter disingenuous arguments. So, straight down to the core of it: what is your real objection to Blue? Why are you so seemingly attached to the current status quo which gives spammers carte blanche and leaves so many struggling to clear their inboxes in the morning? I’d be really interested to know. Thanks for reading. I hope you can help me with this, so that I can understand the anti-Blue stance a little better. Thanks.

I feel your pain, but I have had my site email blown into oblivion by a spammer using it as a bogus return. I joined the frog six months later, when in came out.

You may call Blue Security shady, but spammers are downright *expletive* evil.

Tobo, primarily, it’s not a matter of me being “against” Blue Security, or even having some firm position on the manner in which BS chooses to fights its anti-spam fight — it’s a matter of me thinking that their decision to turn the firehose against Six Apart’s network was incredibly idiotic. And that’s something for which I feel that they should be held accountable in some manner. It’s that simple.

(Note that I’ve also shown a bit of my cards on how I feel about their anti-spam methods, though — I think that they’re dangerously vigilante-like in their implementation, doomed to failure in their action, and all too easy to exploit to cause collateral damage along the way. Those who’ve read me for any amount of time know how I feel about spam itself; I fought the spam fight well before it was even called spam, and remember when you were able to call the ISPs of each and every spammer whose detritus reaached your mailbox in a day and get them shut down. So this doesn’t have anything to do with me defending the spammers or being oblivious to the world around me — the world that causes my mailserver to filter nearly 5K spam messages a day — and everything to do with me not wanting to see irresponsible companies escalate a fight that they can’t win and that will cause all of us to lose in the interim.)

Rather than get dragged into the debate surrounding Blue Security, I want to focus my comment on just one aspect of this mess.

I will readily stipulate that the attacker’s actions are wrong. I feel compelled to point out, however, that despite the way some of the other commenters appear to see the world, this does not mean that sole and total blame must be laid at the feet of the attackers and that Blue is a helpless victim. Even when confronted by a DDoS attack, there are actions I can choose to take in response that are *also* wrong.

Blue’s initial response, to redirect their DNS entry for the servers under attack to their blogsite, was poorly thought out whether or not you support their spam-fighting methodology or not. Tim states: “How COULD blue know that six apart would not be up to the volume?” Therein lies the crux of the issue — what Blue could reasonably be expected to know.

They obviously knew they were under some sort of attack, because they took active measures to try to work around the effects. Those active measures included modifying their DNS records so that browsers trying to contact their main webserver would instead head to their Typepad-hosted blog. Whatever their intentions, this was irresponsible and unethical — especially if, as Tim says, they had no way to know the effects of their actions. Were the inbound attacks making use of the DNS service? I suspect they didn’t know. What was the total volume of the incoming attacks? Again, I doubt they knew (or had any way to accurately assess that information). Was Six Apart’s network configured in such a way to handle that traffic? At the time, again, I doubt they knew. The answer, in hindsight, was clearly “No.”

How is their action any different than overloading a hosting provider with wildly popular traffic? Simple: *they knew the traffic was being generated with hostile intent*. They *knew* they were under attack. To knowingly shift that traffic to a third-party network means *they knowingly redirected an attack to a third party and put that third party’s network at risk. I doubt their contract gives them the right to do that. This wasn’t legitimate web traffic, remember; this was traffic that had already knocked down their server.

Okay, for the sake of argument, maybe they didn’t think that the incoming attacks were performing DNS lookups. Maybe they thought the incoming packets were aimed solely at their IP address. Well and good, but that’s a guess on their part. Maybe the attacking machines were in fact using DNS lookups; maybe somebody behind the attack was monitoring just for those kind of changes and added the new IP addresses to the targeting list. Regardless of that fact, they took an action that they had to know had a high likelihood of spreading the damage.

Tim states that the “blue founders have long and legitimate security backgrounds.” Well, Tim, one of the first things you learn when you’re doing computer security as a professional is that it’s wrong to take actions that cause collateral damage, or that are highly likely to cause collateral damage. So either Blue *knew* that changing the DNS records would redirect the attack, or they were ignorant of the probable results of their actions. In the former event they’ve proven themselves to be malicious; in the latter, not nearly as skilled and experienced as their PR wants you to think.

In fact, it’s worse than that. Blue is no stranger to the amount of controversy that their business model has generated, and they clearly know that spammers aren’t likely to be dissuaded from illegal actions. Why didn’t they take just a minute to realize that somewhere along the way they were going to piss someone off, figure out what the likely responses would be, and do some basic contigency planning so that when they inevitably found themselves in an escalation, they had a plan to follow and extra resources to use?

And lest anyone point out that redirecting www.bluesecurity.com to Typepad represents that kind of planning, I refer again to the basic flaws in their response. If this is the best planning they can come up with — shifting the attack to a third party network, thus knocking that network off the air (apparently with no prior warning, I might add) — then I have serious doubts about their ability to *usefully* fight spammers in the long term.

Jason,

I am exactly one degree away from my ‘source’ in this. I worked with the CTO of BS for 6 years, and have had several conversations with him over the last several days. You can believe what you want as to this authenticity. If you need proof, you have my email address. I wont go into it in more detail in public. I spoke at length with him today about six apart. When they changed the DNS entry to point to their blog, they had ALREADY been blackholed. Noone could get to their IP addresses on the Internet except from with Israel. At the time of the change, they were no longer receiving any of the DDOS, because no one could REACH the IP address in question. For you to repeatedly - judge - by calling the move idiotic, is idiotic. They had no way of knowing at the time the attack was still ongoing. They were under the impression that the attack methods had changed from a DDOS to a “backbone subversion” attack. That term by the way was used by the source of the DDOS, as he taunted blue with what he was going to do next over ICQ. He posted an exchange between him and a network engineer at a teir 1 US ISP, in which it appeared the engineer was agreeing to blackhole their IPs. Once their IPs were off the net, they made the change to that IP address to update their customers. It wasnt until DNS changes propogated over the next couple of hours that the DDOS resumed against six apart. It is these details that Six Apart considered, which you did not know, that made Six Apart choose to work with Blue and not take it as just them shifting the attack. Once Six Apart realized the destination of the attack and contacted blue, blue worked to change the address to localhost. This was not malicious. But again - you can choose to believe what you want.

As for you finding my comments on Blue submitting thousands of false orders through spammers and pfishers websites, I dont understand your comment… maybe you can explain what you feel priceless about it.

Tim, I’m not going to continue fisking all the comments on this post, mostly because I’m working tonight and have other places my attention is needed. (And I repeat from before — “desk job”?) I’ll say, though, that that’s a bit of interesting information about what might have happened; it’d be more interesting (honestly!) if it were substantiated by any of the network providers who were involved in the incident, or even the people at Blue Security themselves.

(The “priceless” comment was directed at the fact that you don’t seem to consider an onslaught of bogus/nonsense email to be a denial of service attack.)

Jason,

My first post wasn’t directed to you - but to the larger YOU of those who were criticizing from afar. I was also under a request by my friend to keep some of the information private. So the desk job comment was not specifically aimed at you - sorry. Most of the network providers are licking their wounds. My friend told me that as near as they can tell, it was 3+ GB per second of load - and according to the spammer it was a mere 8% of what he could deliver. Their registrar was contacted by their DNS providers and asked to remove the domain name! That was the response from one of the largest DNS hosting providers here in the US. Ask to have the registry entry removed so their DNS servers could bare the load. The co-los in israel they were using are still licking their wounds. Again - the DDOS stopped when the IPs were blackholed - so even that co-lo was under the impression the attack had stopped. All of this latest info has really only boiled up in the last day or so. There is a lot more to this story that has not been released. The one clue you should really consider is that Six Apart did not blame them. Again - sorry if you were offended on the desk job comment. You can contact me directly for more info when you get time if you are interested….

Sounds like this round of comments needs to fall under the Apple/x86 holy war group.

Tim: It’s nice that you have a source. I’d love for somebody at Blue to practice good customer communications and post what happened. So what if it’s bad PR. People want to know, and any company that has the balls to explain what happened, regardless of what people might think, gets very high marks in my book.

When any of the companies involved wants to come clean and post the details I’ll be happy to hear them. Posted OFFICIALLY.

• Posted by: Joe Becher on May 4, 2006, 10:25 PM

Dude - I am sure they will post. You also have to realize that they are in complete rebuild mode. They are looking at DDOS solutions that allow them to connect again… I dont think anyone could have predicted the extent of attack they received. I kreep reading posts like - “what did they think would happen?”… I am sure noone would have considered the scale of attack the were under…

(FYI, I’ve purposely not approved a few comments that came in tonight that clearly had faked email addresses and fit the profile of “confrontational, rude, or personal attacks” — if you’re one of the people who made the comments, sorry ‘bout that, but you gots to play by the rules. Alas. And all of them were of the “spammers must all DIE!” variety, which is funny seeing as they all were cloaking their identity and spewing unwelcomely, just like the enemy they purport to be fighting. What was it that Pogo said?)

Tim says “I dont[sic] think anyone could have predicted the extent of attack they received.”

Really? That’s funny, because it doesn’t take a rocket scientist. For *how* many years now have DDoS attacks been around? For *how* many years have they been knocking high-end operations off the net? Many of the previous high-profile (and high-budget) DDoS victims haven’t even been directly attacking spammers!

How could anyone who
a) knows that spammers are using very large botnets…
b) knows that botnets are used for DDoS attacks…
c) knows that spammers will launch DDoS attacks…
and d) knows that the sheer scope of these attacks has only been increasing, enough to hurt some pretty big targets…
*not* have seen this coming?

They didn’t have to know the precise details in advance in order to do some basic 2+2=4 and realize that the inevitable shitstorm was going to large and distinguished, and that they’d better have a plan in place to deal with it.

We’re supposed to be all shocked and horrified that, once they’ve spit in the spammers eyes and said “Do your worst?”, the spammers did just that? Not at all — and if that was the end of the story, I for one would be all up in arms about it. The folks running the botnets are bullies and crooks, pure and simple — but the folks at Blue are appearing more and more like the guys who claim to be protectors from the bullies, go and pick a fight, and get everyone’s ass kicked because they didn’t think it through before issuing their challenge.

• Posted by: Devin L. Ganger on May 5, 2006, 3:02 AM

Thanks Jason. That’s fair enough. So, let me get this right: essentially you are saying that you are of the defeatist frame of mind? You are saying: “I have seen it all before and Blue can’t change things - only make things worse. This is how it is, and it isn’t going to change…” And you mix in some ethics to support your view? I think that’s what you are saying, isn’t it?

The Dutch have a saying, which is: “No is what you’ve already got.” What they mean by this is many people have a mindset which stops them even expecting a Yes in their lives. So they continue expecting No, and don’t even know how to ask for a Yes. The phrase “doomed to failure” has an inherent fatalism in it which is of the No mindset. Many don’t accept No. Fate is, in this instance, not in the hands of a supernatural force controlling the way the world spins, but ours and the spammers. There are more of us than them. I would think that if enough people said “Yes” to themselves and to the day when they woke up, they might consider it possible to change their supposed fates. But there it is: you’ve answered my question, and I take my hat off to you for your clarity.

As for the redirecting of the DDOS to Six Apart, well, I agree with you that it was ill conceived. The dust is still settling at the moment and no explanations have been forthcoming from Blue. Now, there are several interpretations of this action available at the moment: 1) In a panic, Blue redirected the attack to save their servers. 2) Blue assumed that Six Apart’s servers were big enough to handle it. 3) They wanted to affect a US company to get more media attention (difficult to believe, but let’s wait and see). Once Blue is up and running again, we’ll see whether any of these interpretations are correct. It may be, as you state, that Blue was irresponsible. But I suspect that you don’t have enough data to make that call just yet in a truly informed way. And as far as I can see, one error of judgment under stress doees not discredit the entire Blue Frog project. I think what is happening is that you have switched to your default anti-Blue mode, rather than weigh all the evidence - though if you have more information than you are letting on, that would be great to hear.

I remain fascinated by all of this: especially the inherent defeatist mindset amongst many bloggers which almost sides with the spammers and says: “Yeah, these guys had it coming,” and which seems to precede any problems they may have caused Six Apart.

I am reminded of seeing David Blaine in London, in his glass case a few years ago. The mindset of Londoners was irrationally hostile to him. Here was a Yank getting above himself. How dare he come to Britain to try something new? So, many Londoners stood to jeer from below, and actually hurl food at the case. It was like seeing someone in the stocks in mediaeval London. Then there was the whole incident of someone flying a Big Mac up to him in a remote control helicopter… They wanted him to fail…

This is what puzzles me about the Net response to Blue Security. Even your use of the initials BS implies the word Bull Shit. I don’t know whether that is intentional or unsconscious on your part. Why do so many people actually want Blue to fail? Surely not so they can just stand around and say: “I told you so?” I mean, is that really satisfaction enough?

See you,

Tobo.

Devin,

merits of BS tactics aside, judging by the collateral damage on this attack, it was pretty unusual in scale.

I’m intrigued by the phrase “sophisticated DDoS”, isn’t DDoS unsophisticated by definition?

Perhaps it is wise to with hold judgement till the full facts are known.

Hi Devin, your bully at school analogy is interesting, but could you explain to me how Blue Frog has managed to get “everybody’s” asses / arses kicked? As far as I can see, the two main victims of the DDOS attack are Blue Frog and - possibly as a result of Six Apart’s agreement to accept the redirect, possibly not - Six Apart, for a few hours. Blue might have made a mistake, or Blue and Six might have made the mistake together. We don’t know yet. What I do know is that I have been receiving my email and accessing the web normally for the last few days, and servers around the world are managing the extra load. So, why the righteous indignation? No one has died, as far as I know. I haven’t even had the threatened increase in spam I was meant to be getting as punishment for being a Blue Community member, except for a spike over the weekend. So, which “everybody” are you talking on behalf of?

What is needed is a degree of cool headed determination to overcome. God, I sound like a 60s anti-‘Nam protester. But the war analogy is apt in one way: this is a war in which the Blue Community is engaged. The role of those involved in it should be to stay calm headed, cool thinking and determined. Blue may have made a mistake, but this is only one battle. There will be plenty more. Having focus and an expectation of winning are the only things needed to see things through. Blue is off line at the moment, but I am saving all my spams for the moment the Frog goes active. I just want to post my displeasure at being spammed back to the spammer. We all deserve a right to reply, and that is what the Blue Frog is doing. I thought those in the US would be more appreciative of that…

As for your arse: I am sorry about it. Mine is fine, however…

See ya, Tobo.

‘I am sure noone would have considered the scale of attack the were under.’

Uh, no, as Devin said, this is most definitely not rocket science, especially
since DDOSes of this scale have been going on for years. Avoiding it is pretty
easy to do with a little infrastructural preparation, and a failure to do this
makes BS look amateurish.

No matter, though — this scenario fits in nicely with their ‘fight until the
end’ rhetoric, so I’m sure they don’t mind all that much really while they get
to put out press alleging that Tier 1 ISPs are screwing with them…

• Posted by: jm on May 5, 2006, 6:19 AM

Jesus, Tobo, I’m actually not sure your sophistry could have gotten my position more incorrect. I’ll repeat once more that I don’t have any intrinsic position on Blue Security as a company itself (despite your “default anti-Blue” claim), and leave to your own research (again!) how I feel about the entire spam issue, and why I find their particular approach so flawed. Suffice it to say that you couldn’t be more wrong in pretty much all of your assumptions and assertions.

Instead of re-directing DNS requests to another host, just black-hole the request with a non-routable destination address.

Hello Jason and all, well, if u haven’t seen it yet, bluesec has a press release about what happened: http://tinyurl.com/m26pj and the main page of their site is up and running, with some info about it.

i believe the truth about all this is somewhere in the middle. but i personally don’t see anything wrong with bluesec tactics againts the spammers. but if they did messed up the blog company, then should pay for it.

Just as an aside, I haven’t seen anyone here that seems to understand how Blue works. I’ll try to summarize.

1) User gets a spam email and reports it to Blue.

2) The email is analyzed, and the source and links are determined. A human looks at the result and researches which are spam-related, and which are not.

3) The spam-related sources are contacted, and asked to filter their address lists using Blue’s tools.

4) After some length of time, Blue’s opt-out mechanism is activated, meaning that when a user reports another spam from a defined source, instructions are sent out to his Blue Frog client to submit an opt-out request to the spamming website.

There is no onslaught of emails, no DDoS attack, no fake orders. They just use the order form to file an opt-out request (If Blue can find a valid opt-out form, they’ll use that, but fat chance). One opt-out request for every spam recieved by a Blue user. A perfectly proportional response.

All Blue Security is doing is providing a service to their users, to let them easily submit opt-out requests in accordance with the Can-Spam act. All the spammers have to do to stop it is to stop sending spam to Blue users.

Jason,

I totally understand your anger at having SixApart’s site taken down, but you’re blaming the wrong people: Blue Security are victims of a DDoS attack orchestrated by criminals.

Rather than comparing Blue Security to someone redirecting a water pipe to their neighbor’s basement, why not think of them as someone trying to fight the neighborhood bully? They received a blow, ducked, and the punch hit the person behind them.

Because that’s a silly and false analogy, Ron — the folks at Blue Security made an active choice to redirect their main hostname to Six Apart’s network. And if you read, I’ve acknowledged multiple times that the originators of the DDoS were ultimately, fundamentally bad — but that BS had a specific, active role in what then happened to Six Apart’s network.

It seems to me that until people *know* for sure what Blue Security’s intent was when they redirected their address, it is all speculation that says more about the speculator than anything. Blue Security’s actions can be interpreted just as easily in an innocent way as in a hostile way; the analogy of the water leak in the basement is flawed, too. A much more accurate one is that someone *else* stuck a sewer pipe in there and was filling your basement with raw sewage. It is just as accurate to say Blue Security “merely” pumped the sewage into the yard where it flowed into the neighbour’s basement due to bad grading of the lawn.

I don’t use Blue Security’s stuff at all, either.

It has been stated that the DDoS attack had ceased at the time that they switched the DNS. They were just looking for a new home. The spammers eventually found that new home, and attacked it in turn.

The hose analogy is poor, because it suggests that Blue Security can control the hose. The spammers control the hose… they just point it where ever Blue Security goes. What would you have them do? Unless you want to hand a victory to the spammers, they have to go somewhere. They chose one of the biggest, strongest houses they could find… TypePad and LiveJournal host tons of sites and get tons of traffic.

• Posted by: Mark J on May 6, 2006, 5:41 AM

Comments are now being turned off; at this point, I’m just getting people cut-and-pasting press releases in total, or people daring me to publish their “scathing” comments about my morals and intelligence and using fake email addresses while doing so. Ah, the wisdom of the ‘net.