diff --git a/docs/server/kb/linux-setting-memlock.mdx b/docs/server/kb/linux-setting-memlock.mdx index 0659993073..e0d7c1deb8 100644 --- a/docs/server/kb/linux-setting-memlock.mdx +++ b/docs/server/kb/linux-setting-memlock.mdx @@ -1,7 +1,7 @@ --- title: "Linux: Setting memlock when using encrypted database" sidebar_label: "Linux: Setting `memlock` when using encrypted database" -description: "Set the memlock ulimit on Linux to allow RavenDB's Voron storage engine to lock memory pages and prevent swapping for optimal performance." +description: "Configure the memlock ulimit on Linux so that RavenDB can lock memory pages required by encrypted databases." sidebar_position: 4 --- @@ -11,20 +11,66 @@ import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; import LanguageSwitcher from "@site/src/components/LanguageSwitcher"; import LanguageContent from "@site/src/components/LanguageContent"; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; # Linux: Setting memlock when using encrypted database -Encrypted database uses extensively sodium library which requires high values of locked memory limits. -`memlock` refers to memory that will not be paged out, and its limit can be viewed using `ulimit -l`. -The modification of `memlock` limit settings can be achieved by in running session with `prlimit`: -Example, for 1MB limit: + + +* When [encryption at rest](../security/encryption/encryption-at-rest.mdx) is enabled, RavenDB uses the libsodium library to lock certain memory regions in order to prevent plaintext data from being swapped to disk. + Linux imposes a per-process limit on how much memory the process can lock (`memlock`). + If this limit is too low, RavenDB cannot open the encrypted database and throws an `InsufficientMemoryException`. + +* The exception message includes the exact `prlimit` command needed to raise the limit, with the required size in bytes already filled in. + +* In this article: + * [Raising the limit for a running process](linux-setting-memlock.mdx#raising-the-limit-for-a-running-process) + * [Setting the limit persistently](linux-setting-memlock.mdx#setting-the-limit-persistently) + + + + + +Run the command shown in the exception message, replacing `` with the RavenDB process ID +and `` with the value from the error: + +```bash +sudo prlimit --pid --memlock= ``` -prlimit -p pid --memlock 1MB:1MB + +
+ +This takes effect immediately but does not persist across restarts. + + + + + +Add the following to `/etc/security/limits.conf`: + +```text +* soft memlock unlimited +* hard memlock unlimited ``` -Persistent settings can be achieved by adding to `/etc/security/limits.conf` the following: +
+ +Log out and back in (or restart the service) for the change to take effect. +Verify the new limit with: + +```bash +ulimit -l ``` -* soft memlock 1000 -* hard memlock 1000 + + +If RavenDB runs as a `systemd` service, set the memory lock limit in the service unit file instead of `limits.conf`: + +```ini +[Service] +LimitMEMLOCK=infinity ``` + + + diff --git a/docs/server/security/common-errors-and-faq.mdx b/docs/server/security/common-errors-and-faq.mdx index 72602a0845..34f9f0cabf 100644 --- a/docs/server/security/common-errors-and-faq.mdx +++ b/docs/server/security/common-errors-and-faq.mdx @@ -513,7 +513,7 @@ In **Linux**, it is the admin's responsibility to configure higher limits manual -To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. - +To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. +For persistent and service-level configuration, see [Linux: Setting memlock when using encrypted database](../kb/linux-setting-memlock.mdx). diff --git a/docs/server/security/encryption/encryption-at-rest.mdx b/docs/server/security/encryption/encryption-at-rest.mdx index 37bcda9444..2894c002af 100644 --- a/docs/server/security/encryption/encryption-at-rest.mdx +++ b/docs/server/security/encryption/encryption-at-rest.mdx @@ -35,14 +35,14 @@ Once a request is made, RavenDB will start a transaction (either read or write) request, and when the transaction is finished, modified pages are encrypted and written back to the datafile. The data is [locked](../../../server/security/encryption/encryption-at-rest.mdx#locking-memory) during the transaction by default, and it is zeroed when the transaction completes. - + 1. RavenDB makes sure that **no data is written to disk as plain text**. It will always be encrypted. 2. Indexed fields (the actual terms and values being indexed) will reside in memory as plain text. 3. Data of the current transaction will reside in memory as plain text and only for the duration of the transaction. When the transaction ends, the used memory is safely zeroed. 4. Loading documents from the database (using the Studio, the Client API, REST API) means that they will be decrypted to plain text on the server and then sent to the client (securely) by HTTPS. Once the data is received on the client side it is no longer encrypted. RavenDB does not provide encryption on the client side. - + Indexing is most efficient when it is performed in the largest transactions possible. However, using encryption can be very memory intensive, and if memory runs out before the transaction completes, the entire transaction will fail. To avoid this, you can limit the size of indexing batches in encrypted databases using [Indexing.Encrypted.TransactionSizeLimitInMb](../../../server/configuration/indexing-configuration.mdx#indexingencryptedtransactionsizelimitinmb). @@ -77,7 +77,8 @@ In **Linux**, it is the admin's responsibility to configure higher limits manual -To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. +To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. +For persistent and service-level configuration, see [Linux: Setting memlock when using encrypted database](../../kb/linux-setting-memlock.mdx). ## What about Encryption in Transit? diff --git a/versioned_docs/version-6.2/server/kb/linux-setting-memlock.mdx b/versioned_docs/version-6.2/server/kb/linux-setting-memlock.mdx index 2dd242e5d8..c969c2b0ab 100644 --- a/versioned_docs/version-6.2/server/kb/linux-setting-memlock.mdx +++ b/versioned_docs/version-6.2/server/kb/linux-setting-memlock.mdx @@ -10,20 +10,66 @@ import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; import LanguageSwitcher from "@site/src/components/LanguageSwitcher"; import LanguageContent from "@site/src/components/LanguageContent"; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; # Linux: Setting memlock when using encrypted database -Encrypted database uses extensively sodium library which requires high values of locked memory limits. -`memlock` refers to memory that will not be paged out, and it's limit can be viewed usign `ulimit -l`. -The modification of `memlock` limit settings can be achieved by in running session with `prlimit`: -Example, for 1MB limit: + + +* When [encryption at rest](../security/encryption/encryption-at-rest.mdx) is enabled, RavenDB uses the libsodium library to lock certain memory regions in order to prevent plaintext data from being swapped to disk. + Linux imposes a per-process limit on how much memory the process can lock (`memlock`). + If this limit is too low, RavenDB cannot open the encrypted database and throws an `InsufficientMemoryException`. + +* The exception message includes the exact `prlimit` command needed to raise the limit, with the required size in bytes already filled in. + +* In this article: + * [Raising the limit for a running process](linux-setting-memlock.mdx#raising-the-limit-for-a-running-process) + * [Setting the limit persistently](linux-setting-memlock.mdx#setting-the-limit-persistently) + + + + + +Run the command shown in the exception message, replacing `` with the RavenDB process ID +and `` with the value from the error: + +```bash +sudo prlimit --pid --memlock= ``` -prlimit -p pid --memlock 1MB:1MB + +
+ +This takes effect immediately but does not persist across restarts. + + + + + +Add the following to `/etc/security/limits.conf`: + +```text +* soft memlock unlimited +* hard memlock unlimited ``` -Persistant settings can be achieved by adding to `/etc/security/limits.conf` the following: +
+ +Log out and back in (or restart the service) for the change to take effect. +Verify the new limit with: + +```bash +ulimit -l ``` -* soft memlock 1000 -* hard memlock 1000 + + +If RavenDB runs as a `systemd` service, set the memory lock limit in the service unit file instead of `limits.conf`: + +```ini +[Service] +LimitMEMLOCK=infinity ``` + + + diff --git a/versioned_docs/version-6.2/server/security/common-errors-and-faq.mdx b/versioned_docs/version-6.2/server/security/common-errors-and-faq.mdx index d63ca01f6c..35f1d89b34 100644 --- a/versioned_docs/version-6.2/server/security/common-errors-and-faq.mdx +++ b/versioned_docs/version-6.2/server/security/common-errors-and-faq.mdx @@ -512,7 +512,7 @@ In **Linux**, it is the admin's responsibility to configure higher limits manual -To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. - +To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. +For persistent and service-level configuration, see [Linux: Setting memlock when using encrypted database](../kb/linux-setting-memlock.mdx). diff --git a/versioned_docs/version-6.2/server/security/encryption/encryption-at-rest.mdx b/versioned_docs/version-6.2/server/security/encryption/encryption-at-rest.mdx index d87f13b698..ce1bed05c9 100644 --- a/versioned_docs/version-6.2/server/security/encryption/encryption-at-rest.mdx +++ b/versioned_docs/version-6.2/server/security/encryption/encryption-at-rest.mdx @@ -34,14 +34,14 @@ Once a request is made, RavenDB will start a transaction (either read or write) request, and when the transaction is finished, modified pages are encrypted and written back to the datafile. The data is [locked](../../../server/security/encryption/encryption-at-rest.mdx#locking-memory) during the transaction by default, and it is zeroed when the transaction completes. - + 1. RavenDB makes sure that **no data is written to disk as plain text**. It will always be encrypted. 2. Indexed fields (the actual terms and values being indexed) will reside in memory as plain text. 3. Data of the current transaction will reside in memory as plain text and only for the duration of the transaction. When the transaction ends, the used memory is safely zeroed. 4. Loading documents from the database (using the Studio, the Client API, REST API) means that they will be decrypted to plain text on the server and then sent to the client (securely) by HTTPS. Once the data is received on the client side it is no longer encrypted. RavenDB does not provide encryption on the client side. - + Indexing is most efficient when it is performed in the largest transactions possible. However, using encryption can be very memory intensive, and if memory runs out before the transaction completes, the entire transaction will fail. To avoid this, you can limit the size of indexing batches in encrypted databases using [Indexing.Encrypted.TransactionSizeLimitInMb](../../../server/configuration/indexing-configuration.mdx#indexingencryptedtransactionsizelimitinmb). @@ -76,7 +76,8 @@ In **Linux**, it is the admin's responsibility to configure higher limits manual -To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. +To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. +For persistent and service-level configuration, see [Linux: Setting memlock when using encrypted database](../../kb/linux-setting-memlock.mdx). ## What about Encryption in Transit? diff --git a/versioned_docs/version-7.0/server/kb/linux-setting-memlock.mdx b/versioned_docs/version-7.0/server/kb/linux-setting-memlock.mdx index 2dd242e5d8..c969c2b0ab 100644 --- a/versioned_docs/version-7.0/server/kb/linux-setting-memlock.mdx +++ b/versioned_docs/version-7.0/server/kb/linux-setting-memlock.mdx @@ -10,20 +10,66 @@ import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; import LanguageSwitcher from "@site/src/components/LanguageSwitcher"; import LanguageContent from "@site/src/components/LanguageContent"; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; # Linux: Setting memlock when using encrypted database -Encrypted database uses extensively sodium library which requires high values of locked memory limits. -`memlock` refers to memory that will not be paged out, and it's limit can be viewed usign `ulimit -l`. -The modification of `memlock` limit settings can be achieved by in running session with `prlimit`: -Example, for 1MB limit: + + +* When [encryption at rest](../security/encryption/encryption-at-rest.mdx) is enabled, RavenDB uses the libsodium library to lock certain memory regions in order to prevent plaintext data from being swapped to disk. + Linux imposes a per-process limit on how much memory the process can lock (`memlock`). + If this limit is too low, RavenDB cannot open the encrypted database and throws an `InsufficientMemoryException`. + +* The exception message includes the exact `prlimit` command needed to raise the limit, with the required size in bytes already filled in. + +* In this article: + * [Raising the limit for a running process](linux-setting-memlock.mdx#raising-the-limit-for-a-running-process) + * [Setting the limit persistently](linux-setting-memlock.mdx#setting-the-limit-persistently) + + + + + +Run the command shown in the exception message, replacing `` with the RavenDB process ID +and `` with the value from the error: + +```bash +sudo prlimit --pid --memlock= ``` -prlimit -p pid --memlock 1MB:1MB + +
+ +This takes effect immediately but does not persist across restarts. + + + + + +Add the following to `/etc/security/limits.conf`: + +```text +* soft memlock unlimited +* hard memlock unlimited ``` -Persistant settings can be achieved by adding to `/etc/security/limits.conf` the following: +
+ +Log out and back in (or restart the service) for the change to take effect. +Verify the new limit with: + +```bash +ulimit -l ``` -* soft memlock 1000 -* hard memlock 1000 + + +If RavenDB runs as a `systemd` service, set the memory lock limit in the service unit file instead of `limits.conf`: + +```ini +[Service] +LimitMEMLOCK=infinity ``` + + + diff --git a/versioned_docs/version-7.0/server/security/common-errors-and-faq.mdx b/versioned_docs/version-7.0/server/security/common-errors-and-faq.mdx index d63ca01f6c..35f1d89b34 100644 --- a/versioned_docs/version-7.0/server/security/common-errors-and-faq.mdx +++ b/versioned_docs/version-7.0/server/security/common-errors-and-faq.mdx @@ -512,7 +512,7 @@ In **Linux**, it is the admin's responsibility to configure higher limits manual -To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. - +To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. +For persistent and service-level configuration, see [Linux: Setting memlock when using encrypted database](../kb/linux-setting-memlock.mdx). diff --git a/versioned_docs/version-7.0/server/security/encryption/encryption-at-rest.mdx b/versioned_docs/version-7.0/server/security/encryption/encryption-at-rest.mdx index d87f13b698..ce1bed05c9 100644 --- a/versioned_docs/version-7.0/server/security/encryption/encryption-at-rest.mdx +++ b/versioned_docs/version-7.0/server/security/encryption/encryption-at-rest.mdx @@ -34,14 +34,14 @@ Once a request is made, RavenDB will start a transaction (either read or write) request, and when the transaction is finished, modified pages are encrypted and written back to the datafile. The data is [locked](../../../server/security/encryption/encryption-at-rest.mdx#locking-memory) during the transaction by default, and it is zeroed when the transaction completes. - + 1. RavenDB makes sure that **no data is written to disk as plain text**. It will always be encrypted. 2. Indexed fields (the actual terms and values being indexed) will reside in memory as plain text. 3. Data of the current transaction will reside in memory as plain text and only for the duration of the transaction. When the transaction ends, the used memory is safely zeroed. 4. Loading documents from the database (using the Studio, the Client API, REST API) means that they will be decrypted to plain text on the server and then sent to the client (securely) by HTTPS. Once the data is received on the client side it is no longer encrypted. RavenDB does not provide encryption on the client side. - + Indexing is most efficient when it is performed in the largest transactions possible. However, using encryption can be very memory intensive, and if memory runs out before the transaction completes, the entire transaction will fail. To avoid this, you can limit the size of indexing batches in encrypted databases using [Indexing.Encrypted.TransactionSizeLimitInMb](../../../server/configuration/indexing-configuration.mdx#indexingencryptedtransactionsizelimitinmb). @@ -76,7 +76,8 @@ In **Linux**, it is the admin's responsibility to configure higher limits manual -To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. +To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. +For persistent and service-level configuration, see [Linux: Setting memlock when using encrypted database](../../kb/linux-setting-memlock.mdx). ## What about Encryption in Transit? diff --git a/versioned_docs/version-7.1/server/kb/linux-setting-memlock.mdx b/versioned_docs/version-7.1/server/kb/linux-setting-memlock.mdx index 2dd242e5d8..c969c2b0ab 100644 --- a/versioned_docs/version-7.1/server/kb/linux-setting-memlock.mdx +++ b/versioned_docs/version-7.1/server/kb/linux-setting-memlock.mdx @@ -10,20 +10,66 @@ import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; import LanguageSwitcher from "@site/src/components/LanguageSwitcher"; import LanguageContent from "@site/src/components/LanguageContent"; +import Panel from "@site/src/components/Panel"; +import ContentFrame from "@site/src/components/ContentFrame"; # Linux: Setting memlock when using encrypted database -Encrypted database uses extensively sodium library which requires high values of locked memory limits. -`memlock` refers to memory that will not be paged out, and it's limit can be viewed usign `ulimit -l`. -The modification of `memlock` limit settings can be achieved by in running session with `prlimit`: -Example, for 1MB limit: + + +* When [encryption at rest](../security/encryption/encryption-at-rest.mdx) is enabled, RavenDB uses the libsodium library to lock certain memory regions in order to prevent plaintext data from being swapped to disk. + Linux imposes a per-process limit on how much memory the process can lock (`memlock`). + If this limit is too low, RavenDB cannot open the encrypted database and throws an `InsufficientMemoryException`. + +* The exception message includes the exact `prlimit` command needed to raise the limit, with the required size in bytes already filled in. + +* In this article: + * [Raising the limit for a running process](linux-setting-memlock.mdx#raising-the-limit-for-a-running-process) + * [Setting the limit persistently](linux-setting-memlock.mdx#setting-the-limit-persistently) + + + + + +Run the command shown in the exception message, replacing `` with the RavenDB process ID +and `` with the value from the error: + +```bash +sudo prlimit --pid --memlock= ``` -prlimit -p pid --memlock 1MB:1MB + +
+ +This takes effect immediately but does not persist across restarts. + + + + + +Add the following to `/etc/security/limits.conf`: + +```text +* soft memlock unlimited +* hard memlock unlimited ``` -Persistant settings can be achieved by adding to `/etc/security/limits.conf` the following: +
+ +Log out and back in (or restart the service) for the change to take effect. +Verify the new limit with: + +```bash +ulimit -l ``` -* soft memlock 1000 -* hard memlock 1000 + + +If RavenDB runs as a `systemd` service, set the memory lock limit in the service unit file instead of `limits.conf`: + +```ini +[Service] +LimitMEMLOCK=infinity ``` + + + diff --git a/versioned_docs/version-7.1/server/security/common-errors-and-faq.mdx b/versioned_docs/version-7.1/server/security/common-errors-and-faq.mdx index 63f4b41cf8..2e4510052f 100644 --- a/versioned_docs/version-7.1/server/security/common-errors-and-faq.mdx +++ b/versioned_docs/version-7.1/server/security/common-errors-and-faq.mdx @@ -512,7 +512,7 @@ In **Linux**, it is the admin's responsibility to configure higher limits manual -To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. - +To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. +For persistent and service-level configuration, see [Linux: Setting memlock when using encrypted database](../kb/linux-setting-memlock.mdx). diff --git a/versioned_docs/version-7.1/server/security/encryption/encryption-at-rest.mdx b/versioned_docs/version-7.1/server/security/encryption/encryption-at-rest.mdx index d87f13b698..ce1bed05c9 100644 --- a/versioned_docs/version-7.1/server/security/encryption/encryption-at-rest.mdx +++ b/versioned_docs/version-7.1/server/security/encryption/encryption-at-rest.mdx @@ -34,14 +34,14 @@ Once a request is made, RavenDB will start a transaction (either read or write) request, and when the transaction is finished, modified pages are encrypted and written back to the datafile. The data is [locked](../../../server/security/encryption/encryption-at-rest.mdx#locking-memory) during the transaction by default, and it is zeroed when the transaction completes. - + 1. RavenDB makes sure that **no data is written to disk as plain text**. It will always be encrypted. 2. Indexed fields (the actual terms and values being indexed) will reside in memory as plain text. 3. Data of the current transaction will reside in memory as plain text and only for the duration of the transaction. When the transaction ends, the used memory is safely zeroed. 4. Loading documents from the database (using the Studio, the Client API, REST API) means that they will be decrypted to plain text on the server and then sent to the client (securely) by HTTPS. Once the data is received on the client side it is no longer encrypted. RavenDB does not provide encryption on the client side. - + Indexing is most efficient when it is performed in the largest transactions possible. However, using encryption can be very memory intensive, and if memory runs out before the transaction completes, the entire transaction will fail. To avoid this, you can limit the size of indexing batches in encrypted databases using [Indexing.Encrypted.TransactionSizeLimitInMb](../../../server/configuration/indexing-configuration.mdx#indexingencryptedtransactionsizelimitinmb). @@ -76,7 +76,8 @@ In **Linux**, it is the admin's responsibility to configure higher limits manual -To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. +To figure out what the new limit should be, look at the exception thrown by RavenDB, which includes this size. +For persistent and service-level configuration, see [Linux: Setting memlock when using encrypted database](../../kb/linux-setting-memlock.mdx). ## What about Encryption in Transit?