The 16-man team, called CynoSure Prime, sifted through the Ashley Madison source code that was posted online by hackers and found a major error in how passwords were handled on the website.
They claim that this allowed them to crack over 11 million of the 36 million password hashes stored in the website's database, which has also been leaked.
A few weeks ago such a feat seemed impossible because security experts quickly observed from the leaked data that Ashley Madison stored passwords in hashed form -- a common security practice -- using a cryptographic function called bcrypt.
Hashing is a form of one-way encryption. A clear text string, like a password, is run through an algorithm, typically multiple times, in order to generate a unique string of characters that serves as its representation. The procedure is not supposed to be reversible unless the algorithm is flawed.
However, recovering the original password from a hash is sometimes possible by using brute-force methods. This is known as hash cracking and involves running a very large number of possible passwords through the exact same algorithm that was used to generate the original hashes and looking for matches.
The success of such efforts depends on many factors: the type of hashing function used, its implementation, whether additional secret values called salts were added to the passwords, the complexity of the passwords themselves and the hardware resources available to the attackers.
Bcrypt is more computationally intensive than some other functions like MD5, which favors performance over brute-force protection. In addition, the Ashley Madison developers used a cost factor of 12 in their implementation, meaning that each possible password an attacker wants to test needs to be put through 4,096 rounds of hashing.
This makes cracking, even with an average-size dictionary -- a collection of common passwords -- and a very powerful hardware rig, very slow. The larger the dictionary the greater the chance of findings matches, but the slower the process.
A security expert named Dean Pierce made an attempt on the first 6 million Ashley Madison hashes using a list of plain text passwords leaked from game publisher RockYou in 2009. After five days he managed to crack only 4,000 hashes. That's 0.06 percent.
Researchers from antivirus vendor Avast tried as well and let their hash-cracking rig run for two weeks. The result: 26,994 recovered passwords, of which only 1,064 were unique -- used by a single user.
The CynoSure Prime team realized that attempting to brute-force the bcrypt hashes will not get them much further, so they started to look for possible errors in how passwords were handled on the website.
A variable called $loginkey piqued their interest. The team found two locations in the code where it was generated, but in slightly different ways.
In one instance $loginkey was generated upon account creation and was defined as the MD5 hash of two other variables: one holding the username and one holding the bcrypt hash of the user's password.
This made the team wonder if the password variable had always been defined as the password's hash. Digging through old code changes they found that prior to June 2012, the variable was actually using the user's plain text password.
It also turned out that when the Ashley Madison developers later implemented bcrypt hashing, they didn't bother regenerating the loginkey variables for early users.
"This meant that we could crack accounts created prior to this date with simple salted MD5," the team said in a blog post. Also, the old code converted the password to lowercase characters before using it, reducing the number of possible characters in a password to 26 and making it faster to brute-force it, they said.
The second instance of $loginkey generation used a combination of the username, password and email variables, plus a constant. This method of generating the $loginkey was used when a user modified their account attributes -- username, password or email.
However, as in the first case, it hadn't always used the bcrypt password hash as the password variable. This meant that the CynoSure team could now recover passwords for accounts that had been modified prior to the code change in 2012.
By creating rules in their MD5 hash cracking program, the team managed to isolate the securely generated, post-2012, loginkey variables from the insecure ones. Just a few hours later, they had already cracked 2.6 million passwords and after a few days, 11.2 million.
Avid Life Media, the company that owns AshleyMadison.com, did not immediately respond to a request for comment.
The issue, though, poses significant online security risks for a very large number of Ashley Madison users who might have used the same password on other websites and haven't changed it since then. Past breaches have shown that password reuse is rampant on the Internet.
The incident should also serve as a lesson to other developers: When you implement a new security feature in your website or application, make sure that it's applied to everyone, not just new users.