Keeping Elasticsearch data secure from attacks and exposure

In the last few years enterprises have seen an unprecedented amount of data lost from vulnerable Elasticsearch clusters. Tens of billions of records of highly sensitive data across financial services, healthcare, high-tech, retail, and hospitality segments have been exposed or stolen. A simple web search for Elasticsearch data breaches yields an unsettlingly large number of results going back a few years to as recently as a few hours ago.

This article breaks down the fundamental vulnerability with enterprise search platforms and describes how Portal26 provides a seamless plugin that delivers NIST FIPS 140-2 validated searchable encryption-in-use to eliminate this critical gap.

Is Elasticsearch secure?

Though extremely powerful for search, Elasticsearch is vulnerable from a data security standpoint

Enterprise search platforms such as Elasticsearch take in enormous amounts of data and index them in many different ways to enable high-performance search and analytics. For this type of indexing to work towards full featured search, such as prefixes, suffixes, wildcards, ranges, etc. live data (i.e. that is no longer at-rest in underlying file systems) inside Elasticsearch that cannot be encrypted. If encryption is applied to Elasticsearch indexes, they will no longer be able to support search. With the primary purpose of the platform being search, this presents an unfortunate paradox.

In the last few years this tradeoff has been thoroughly exploited by ransomware actors who search for credentials into data rich Elasticsearch environments or crawl the web for misconfigured or inadvertently exposed clusters. Once inside they steal massive amounts of data and proceed to extort victims. In addition to external cyberattacks, Elasticsearch platforms are also vulnerable to insider attacks.

While the underlying platforms are able to encrypt data at-rest (in the file system while not in use) and also encrypt in transit as data moves between nodes (internode TLS), data still remains highly vulnerable inside the indexes themselves. This is the typical breach point.

The Importance of Encryption

Encryption is one of the best data security controls ever created. Since its invention in the late 1970s and with all the improvements since then, encryption is the one mechanism that can stand up to ransomware and other cyber attackers by making stolen data unreadable.

Elasticsearch Security with Portal26

Portal26 utilizes encryption to secure data inside Elasticsearch by extending its umbrella of protection beyond data-at-rest and data-in-transit to include data-in-use. With this new defense in place, sensitive data inside Portal26’s Elasticsearch encryption capabilities enables enterprise search platforms to become immune to compromise at the hands of external attackers, malicious insiders, or accidental exposure. Any data queried or otherwise exfiltrated retains encryption and cannot be used as leverage against the enterprise even if it is accessed using admin privileges.

Portal26 Plugin for Elasticsearch

Elasticsearch data Security Features with Portal26

Elasticsearch data encryption: Portal26 keeps data always encrypted while retaining search

Portal26 offers high performance real-time encryption-in-use for enterprise search platforms such as Elasticsearch. Portal26’s Elasticsearch data encryption stays in place even as data is actively indexed and searched by the platform.

The Elasticsearch security plugin enables the construction of the encrypted search index and also intercepts and transforms all queries against protected data. Both the datastore as well as any applications built on top of it do not need to be aware of the plugin’s existence as all operations are performed transparently and using native platform capabilities.

Portal26 releases data in multiple privacy preserving formats

In addition to retaining encryption within the platform and indices, Portal26 also supports traditional and format preserving encryption, tokenization, masking and redaction of sensitive data when it is released from the platform. The Portal26 Elasticsearch security plugin is a completely transparent mechanism and sensitive data can be protected without losing data usability, searchability, and without modifying either the Elasticsearch platform or applications built on top of it.

Portal26 supports regulatory compliance and audit

By utilizing NIST FIPS 140-2 validated encryption for data in all states (including data while it is actively being used) Portal26 helps protected Elasticsearch Search clusters support FEDRAMP, GDPR, CCPA, ITAR and Data Residency compliance. Portal26 is also an extremely efficient way to support least privilege access to data inside Elasticsearch and applications built on top of it.

Portal26 includes rich BYOK (bring your own key), HYOK (hold your own key) functionality and granular protection

Portal26 offers rich BYOK (Bring your own key) and HYOK (Hold your own key) capabilities for all data secured by Portal26 inside a Portal26 protected Elasticsearch deployment. Portal26 protection is offered at all levels including index and field level protection. Portal26 protected Elasticsearch clusters never see clear text sensitive data as it is already encrypted by the time it reaches the index to be persisted. This makes them highly resistant to common attack patterns, data exfiltration, extortion, and insider attacks. This is further described in a separate section below.

Portal26 provides post attack visibility and evidence of encryption for regulatory compliance and audit

All Portal26 algorithms are NIST CAVP certified and our underlying engine is NIST FIPS 140-2 validated. This allows our customers to enjoy immediate compliance, efficiently apply data privacy controls across the enterprise, and most importantly, it allows Portal26 to provide granular field-level evidence of encryption in the event of an attack. CISOs can rely on Portal26 for visibility on what attackers observed, accessed, and exfiltrated and reporting that can be provided to auditors, regulators, and boards of directors.

Elasticsearch Data Encryption: Portal26’s Key Capabilities

• Encrypt your indexes with FIPS 140-2 encryption
• Perform full-featured search and analytics without any data decryption including in memory or in the reverse index
• Release data in nine secure and private formats including traditional and format preserving encrypted, tokenized, whole or partially masked, redacted, hashed, anonymized, or valet (where third parties can query the data but not decrypt it)
• Protects query parameters
• Grant clear text access on the basis of least privilege
• Deploy BYOK/HYOK at any granularity
• Become immune to breach, ransomware, double and triple extortion, insider attacks and misconfigurations
• Immediately support with data protection requirements for FEDRAMP, HIPAA, PCI, GDPR, CCPA, and other major regulations
• Deploy on all clouds, hybrid, or on-prem
• Supported on Elasticsearch and AWS OpenSearch

Find Out More About Securing Elasticsearch >

How does it work?

For Secure Indexing and Search

The Portal26 Elasticsearch security Plugin for enables sensitive data to be indexed and searched while always keeping the data in FIPS 140-2 validated encryption format:

• All sensitive data is preprocessed and encrypted prior to being indexed.
• Queries are intercepted and reformulated to execute in encrypted space without any data decryption whatsoever. Portal26 supports numerous types of queries including term, prefix, wildcard, match, match-phrase, match-phrase-prefix, range, term (CIDR) etc.
• By default, query results are released in encrypted form. Here is an example of query results:

• Portal26 plugin does not have a significant impact on ingest and search performance. The overhead is typically up to about 10% when ingesting data and 2-3% while searching.

This means that even if attackers find their way to your Elasticsearch deployment, the data they could exfiltrate would be encrypted and unusable.

For Secure Data Release

In order for a legitimate user to get clear text out of Portal26 protected Elasticsearch there are several controlled release processes including direct allow listing as well as controlled release via pre-integrated Portal26 proxy or Portal26 translation service. All release configurations are defined at the granular field-level, and different fieldscans be set up to behave differently.

View Portal26’s Elasticsearch security plugin in action by booking a demo here.

Digging deeper into customer-controlled keys for Elasticsearch (BYOK/HYOK)

The Business Problem

SaaS providers are often asked two basic data security questions:

1. Can you tell us how you intend to protect our data?
2. Can your devops staff see our data?

SaaS providers can point to the encryption built into the storage layer (AWS S3’s SSE-S3 or EBS volume encryption). These encryption solutions protect the data from being viewed by AWS datacenter employees but offer no security or privacy from the SaaS providers own devops team.

SaaS prospects know this very well, especially those in regulated industries such as banking, finance, healthcare. With ransomware attacks and data breaches on the rise, they demand more. This drives SaaS companies to either ship on-prem software or stand up dedicated instances. Neither is desirable from a SaaS standpoint.

Ideally SaaS companies should be able to respond to data protection queries with statements such as:

1. “We encrypt your data with the highest data protection standards (NIST FIPS 140-2).”
2. “Our devops team cannot see your data in clear text.”
3. “As our premium enterprise customer, you can supply and control the encryption keys. You can revoke them whenever you want.” aka BYOK – Bring Your Own Key or HYOK – Hold Your Own Key.

However, implementing BYOK/HYOK on Elasticsearch involves a massive engineering effort and an excellent understanding of specialized encryption.

Portal26 Solution

Portal26 can help.

Portal26 is the preeminent provider of data encryption solutions protecting big data stores such as AWS S3 and search engines such as AWS OpenSearch. All Portal26 offerings come with consistently implemented BYOK/HYOK capabilities giving SaaS companies a simple and seamless key management experience.

Companies can roll out Portal26 and secure their SaaS backend in a matter of weeks with minimal engineering effort allowing SaaS engineering teams to focus on what they do the best – building and running their platform.

BYOK/HYOK in Portal26 for Elasticsearch is built further upon the existing strengths of searchable encryption. It is built using two simple premises – index specific encryption keys and ability to read those individual keys from any location.

As a SaaS operator it requires one thing – providers must maintain a separate search index (or a set of indices) for each customer. This enables independent upgrades, encryption key rotation, etc. Portal26 plugin includes a key registry to manage encryption key locations corresponding to customer specific indexes. Portal26 reads the encryption keys directly from customers keystores and supports common key stores such as AWS Secrets Manager, Hashicorp Vault etc. Adding a keystore integration is easy and we are continuously adding these to the solution.

Steps to enabling data protection with BYOK/HYOK for your Elasticsearch based application.

1. Install Portal26 Plugin for Elasticsearch security.
2. Choose which fields to protect. In a typical Elasticsearch index, not all documents need to be secured and searched. You choose specific fields to secure.
3. Test out the front-end application and ensure it works seamlessly.
4. Create a key registry – specify which index should use which key and provide Portal26 with the credentials (in encrypted form) to access the corresponding keystores.
5. Start loading data and querying it.

Conclusion: No more tradeoffs between search and security

Portal26 enables Elasticsearch to retain its enormous power while making it the most secure search platform in the industry.

To book a demo of Portal26’s Elasticsearch security plugin and to find out more about securing Elasticsearch, please schedule a demo today.

Schedule a Demo >