Kaseya Announces New Service Restoration DateCEO Fred Voccola: 'Company Let Customers Down'
Thousands of organizations that rely on Miami-based Kaseya's VSA software to remotely manage systems are going to have to wait longer to regain the ability to use it.
In his latest video message to customers posted early Thursday, Kaseya CEO Fred Voccola announced further delays as the company patches both the software-as-a-service version of its VSA software, which was not exploited by attackers, and the on-premises version, which was used to infect about 60 of Kaseya's managed service provider customers and up to 1,500 of their clients with ransomware.
Kaseya took its SaaS software offline in the wake of the attack and urged all on-premises users to immediately deactivate the software. The company had hoped to restore its VSA SaaS software by Tuesday, in fully patched form, and to issue patches to users for the on-premises version 24 hours later. But in his latest video statement, the bleary-eyed CEO says the company now hopes to patch the SaaS software by 4 p.m. EDT Sunday.
VSA, which is widely used by managed service providers, facilitates making changes and updates to systems remotely, making it a powerful tool.
Unfortunately, those powers were co-opted by attackers affiliated with the REvil ransomware group, who exploited vulnerabilities in VSA on July 2 and then distributed ransomware to some Kaseya MSP customers and their clients (see: Kaseya: Up to 1,500 Organizations Hit in Ransomware Attack).
Kaseya turned off its SaaS platform shortly after detecting the attack, which was the "nuclear" option, writes Frank Breedijk, a Dutch researcher involved in identifying VSA vulnerabilities in April. But Kaseya didn't have that kind of control for on-premises users, which had to be warned to take VSA offline (see: Kaseya Raced to Patch Before Ransomware Disaster).
Reasons for Delay
"It was my decision to [delay restoration of service] and no one else's," Voccola says in the video. "The vulnerabilities that were exploited in the attack - we had them locked and felt comfortable with the [SaaS] release. But [based on suggestions] of third-party engineers and IT staff, we [implemented] additional layers of security for things we might not be able to foresee."
Voccola says the company's VSA software will be "hardened" prior to patch releases Sunday for both the SaaS and on-premises versions.
"I feel extremely confident that this Sunday, at 4 p.m. Eastern, we'll have customers coming back on," Voccola adds. "The fact that we had to take down VSA is very disappointing to me … I let the community down, let the company down. Our company let you down."
Voccola also announced the creation of a customer assistance program, "Kaseya Cares 2021," supplying direct financial relief to MSPs that have been "crippled" by the REvil cybercriminals.
The CEO says Kaseya will spend millions of dollars to work with third-party consulting companies and its own teams to allow delays of payments for its customers and will provide "other means to address everyone who has been down for the past several days."
On Tuesday, Kaseya outlined security improvements to VSA, including a content delivery network with a web application firewall, along with a 24/7 independent security operations center for every VSA server, each with the ability to quarantine and isolate files and entire VSA servers.
Following the advice of law enforcement agencies - including the Federal Bureau of Investigation and the Department of Homeland Security - Voccola says VSA will be "exponentially more secure" once it's restored.
On Premises Startup Guide
Kaseya published a "Startup Readiness Guide" on Thursday, designed to help users of on-premises VSA servers prepare for the forthcoming patch release containing critical security fixes.
Recommendations from the guide include:
- Ensure the VSA server is isolated. Before powering on the VSA server, Kaseya advises clients to ensure it's isolated from inbound and outbound traffic and segregated from the main network.
- Check the system for indicators of compromise. Run the “compromise detection tool” on the VSA server.
- Patch the operating systems of the VSA servers. This step requires internet access; if the machine is completely isolated from the internet, restore "outbound" internet connectivity. Double-check that the Kaseya Edge Services service is still disabled.
- Use URL rewrite to control access to VSA through IIS. Control access to the VSA server and only allow the necessary access to the system user interface.
- Install FireEye agent. Kaseya is providing free licenses of FireEye Endpoint Security agents for each customer’s VSA server.
- Remove pending scripts/jobs. Customers will be provided with instructions on how to clear any pending VSA scripts/jobs that accumulated since the shutdown to run prior to installing the patch and restarting the server.
SaaS Startup Guide
In a yet another update midday Thursday, Kaseya released a Startup Guide for its SaaS service.
"Kaseya has found no indicators of compromise from the incident and had no reports of compromises for any VSA SaaS customers," the vendor confirms.
Following a restart and login, the vendor recommends SaaS users:
- Review system configurations. Check for accuracy on common items, including number of agents, user accounts, views, reports and policy management.
- Resume an agent group. A "phased approach" is recommended - starting with a smaller segment of machine groups.
- Check VSA after resuming agents. Establish a live connect session with a machine; connect and verify the remote control session operates as expected.
- Resume remaining agents. Restore additional agents to service.