5 signs your Web application has been hacked
Most attacks against Web applications are stealthy and hard to spot. That’s a problem, because once attackers get in, they lurk undetected on networks for an average of 205 days, according to the 2015 Verizon Data Breach Investigations Report. Most organizations find out they’ve been compromised from someone else, such as when they get a call from law enforcement or an irate customer.
How can you tell if your Web application has been hacked “When your Web application is compromised, it will start to do things out of the ordinary,” says Steve Durbin, managing director of Information Security Forum. The key is to gain a thorough understanding of what constitutes normal behavior for your application, then keep your eyes peeled for aberrations.
Here are five signs your Web application has been compromised -- and where to begin your investigation. You'll also find some commonsense advice about securing your Web application, whether or not you've been hacked.
Monitoring applications is the single best way to notice when something suspicious is occurring.
Perhaps the application now takes much longer to render the results page from the database than it used to. Perhaps the application is displaying pages at unexpected times or redirecting users to a different page. Perhaps network traffic has increased, but there's no accompanying marketing campaign to explain the surge. A small Web shop that normally sees about 50 orders a day, for example, should question a day with 5,000 orders.
These are not necessarily indicators a Web application has been compromised, of course. Slow page loads can easily result from temporary connectivity issues -- or even a DDoS attack, if you think attackers would have any reason to launch one. But it’s always better to investigate something screwy right away instead of waiting for a major disturbance.
If the application redirects users to a different page, find out why. Is a malicious ad taking over the page function Has the code on the page been modified recently Has the info in the database been tampered with Regularly interact with the application in the production environment to parse normal behavior so that unexpected behaviors can be flagged right away for immediate investigation.
Logs can be a gold mine of attack information if set up correctly. Sifting through database logs can show unexpected queries and reveal when information is being dumped. If database logs show multiple errors in a short period of time, that may be a sign someone is poking around the application looking for -- or has already found -- a SQL injection vector. Trace back to where the database queries originated and make sure the application is correctly handling inputs.
Your Web server software can log inbound and outbound network connections through system FTP and HTTP logs. (They are turned on, right) Those logs can pick up warning signs of unauthorized or malicious activity.
Web servers should generally only initiate connections with internal databases. If there are outbound network connections from your Web server to public IP addresses, it’s time to ask why. Unexplained file transfers show data is leaving the Web server. That could be a clue that attackers have already siphoned data from the application and are transferring the contents to remote servers.
Don’t be so focused on what’s moving outside the network that you ignore lateral movement. If the Web server is communicating with other internal network resources, such as user file shares and individual user computers, that can be a sign attackers have gained entry and are moving around the network. If the application lets users upload files, then make sure it uses a dedicated file server and not a general one employed within the enterprise, for example.
Much like server logs, application logs can tell you when things go wrong, provided they’ve been set up correctly and are monitored. Make sure the application logs admin-level tasks, such as creating user accounts, or admin accounts. If the application creates admin-level or other privileged accounts, verify the accounts are legitimate -- not established by attackers.
Web applications should also show when administrators are logging in, so you should regularly check for access from unexpected locations and times. Verify what the administrator accounts are doing. Unexplainable instances of Web application administrator account access is typically a strong indicator of a breach.
If there’s an increase in the number of errors related to form submissions or more errors show up when pages are loaded, chances are the application is trying to do something it wasn't designed to. If you notice an increase in errors, trace the page that's triggering the errors and find out what may have changed.
Monitor processes running on the Web server, to detect when the server spawns unknown processes or runs a known process at an unusual time. An unknown process is generally a big clue your application is no longer under your control.
Once an attacker has an account on the server, there is little the attacker can’t do. Regularly monitor the server for users being created, especially those with elevated privileges. Those accounts aren’t typically created on the fly, so it’s worth following up whenever an account is created. If certain users who shouldn’t be requesting elevated privileges or root access are suddenly making those requests, you may be witnessing an attacker use a stolen credential.
Get in the habit of looking at crontabs on Linux servers or Scheduled Tasks on Windows servers and knowing what normal entries look like. If new jobs are added, that could be a clue the application is doing something unexpected. Perhaps it’s only an ad hoc maintenance job -- but it could be an attacker’s attempt to get the application to phone home periodically to get new instructions from the command-and-control server. The attacker could also be sending extracted data in small automated batches to a remote server.
Do the timestamps on your Web application files show files you last changed years ago have been recently edited If the Web server wasn’t correctly configured or the application has a vulnerability, attackers could modify the application to run their own malicious code. This could be injected JavaScript or a rewritten module. Check timestamps to make sure files aren’t being modified without authorization. If the file has been modified, compare the files against an earlier version to find out what changed.
Several utilities can scan the application to look for malicious code. Run them periodically to make sure the changes don't sneak in. (Sucuri is one such tool.)
Are there a lot of new files on the Web server that can’t be accounted for with normal usage New files showing up in the Web root is a problem, especially if they are scripts or other types of executable files. Adding files to Web root should be a thoroughly documented process and never a surprise. If you are finding new files in Web root or elsewhere on the server, then you have a breach. The attacker may be using your application to serve malware to unsuspecting site visitors or running a script redirecting them elsewhere. It could also be a text file containing harvested data.
There have been cases of attackers creating a whole new directory and installing their own application. Instead of touching the actual Web application, they piggyback on the domain and the server to run their own tools.
If the application uses third-party plug-ins, check to make sure plug-ins aren't getting updated or installed without warning. Don't install plug-ins simply because they make your site look cool -- exercise due diligence to make sure the plug-in won't add malicious functionality to the site. Scanning tools such as the one from White Hat Security can help uncover potential attack code.
If your application has been compromised and is actively spreading malware, chances are other security tools have picked up on it. Google is very quick about blocking pages that have a bad reputation among Chrome users; other browsers regularly update their blacklists as well. Regularly check your application from other browsers to see if there are any messages -- or look up your site using Google's Safe Browsing tool.
Monitor social media and help desk emails for complaints from users. If users say they aren't getting password reset emails because the messages are being treated as spam, it's worth investigating whether your application has been flagged as a spam relay.
If issues are found, make a backup of the application and the server so that you can look at it later for forensics. If you’re restoring from backups, make sure you have a clean copy, so you aren’t merely reinstalling the malware. Identify the affected files and replace them with clean ones. Of course, this means backups need to occur on a regular basis.
Once the application has been restored and unnecessary files have been removed, change all of the passwords, including the one for CMS, administrator accounts, and all the individual services. Turn on two-factor authentication where possible and set up VPN access where it isn’t. This makes the application secure and prevents attackers from strolling back in.
Harden the application. Remove write permissions where they’re not needed and never use default passwords. It makes sense to use directory paths that may be harder to guess (don’t merely use /admin for the control panel if you can control that). For PHP applications, it can be as simple as enabling safe mode in the php.ini file. Security scanners will check for known security vulnerabilities in your application.
On your personal or work laptop, you use antivirus software, are careful about what programs you download, and regularly apply updates to your operating system and third-party software. That advice applies to Web servers and applications, too. Regularly update third-party applications to make sure vulnerabilities have been patched. Modern antivirus tools on servers will catch most publicly available Web shells and detect malware being installed on the server.
In almost all the cases, you need to restore the application and close the flaw that allowed the attackers in. While rebuilding the server from scratch and setting up the application is a possibility, it’s also usually saved as the last resort. If you don’t have up-to-date and clean backups, a full refresh may be your only option. But you won't know to fix your applications if you don't regularly look for signs the attackers have already broken in.