mirror of
https://gitlab.computer.surgery/matrix/grapevine.git
synced 2025-12-16 15:21:24 +01:00
make all caches individually configurable
Also: * Removes the `cache_capacity_modifier` option * Renames the `pdu_cache_capacity` option to `cache.pdu`
This commit is contained in:
Charles Hall
parent
9fab7538a0
commit
1148c6004f
4 changed files with 65 additions and 110 deletions
|
|
@ -81,27 +81,26 @@ This will be the first release of Grapevine since it was forked from Conduit
|
|||
database backends.
|
||||
* The latter two commands had poor UX and didn't have any noticable effect on
|
||||
memory consumption.
|
||||
9. **BREAKING:** Remove the `global.conduit_cache_capacity_modifier` and
|
||||
`global.pdu_cache_capacity` configuration options.
|
||||
([!124](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/124))
|
||||
* Instead, it is now possible to configure each cache capacity individually.
|
||||
|
||||
### Changed
|
||||
|
||||
1. **BREAKING:** Rename `conduit_cache_capacity_modifier` configuration option
|
||||
to `cache_capacity_modifier`.
|
||||
([5619d7e](https://gitlab.computer.surgery/matrix/grapevine/-/commit/5619d7e3180661731800e253b558b88b407d2ae7))
|
||||
* If you are explicitly setting this configuration option, make sure to
|
||||
change its name before updating.
|
||||
2. **BREAKING:** Rename Conduit to Grapevine.
|
||||
1. **BREAKING:** Rename Conduit to Grapevine.
|
||||
([360e020](https://gitlab.computer.surgery/matrix/grapevine/-/commit/360e020b644bd012ed438708b661a25fbd124f68))
|
||||
* The `CONDUIT_VERSION_EXTRA` build-time environment variable has been
|
||||
renamed to `GRAPEVINE_VERSION_EXTRA`. This change only affects distribution
|
||||
packagers or non-Nix users who are building from source. If you fall into
|
||||
one of those categories *and* were explicitly setting this environment
|
||||
variable, make sure to change its name before building Grapevine.
|
||||
3. **BREAKING:** Change the default port from 8000 to 6167.
|
||||
2. **BREAKING:** Change the default port from 8000 to 6167.
|
||||
([f205280](https://gitlab.computer.surgery/matrix/grapevine/-/commit/f2052805201f0685d850592b1c96f4861c58fb22))
|
||||
* If you relied on the default port being 8000, either update your other
|
||||
configuration to use the new port, or explicitly configure Grapevine's port
|
||||
to 8000.
|
||||
4. Improve tracing spans and events.
|
||||
3. Improve tracing spans and events.
|
||||
([!11 (a275db3)](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/11/diffs?commit_id=a275db3847b8d5aaa0c651a686c19cfbf9fdb8b5)
|
||||
(merged as [5172f66](https://gitlab.computer.surgery/matrix/grapevine/-/commit/5172f66c1a90e0e97b67be2897ae59fbc00208a4)),
|
||||
[!11 (a275db3)](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/11/diffs?commit_id=a275db3847b8d5aaa0c651a686c19cfbf9fdb8b5)
|
||||
|
|
@ -117,31 +116,31 @@ This will be the first release of Grapevine since it was forked from Conduit
|
|||
[!69](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/69),
|
||||
[!102](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/102),
|
||||
[!127](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/127))
|
||||
5. Stop returning unnecessary member counts from `/_matrix/client/{r0,v3}/sync`.
|
||||
4. Stop returning unnecessary member counts from `/_matrix/client/{r0,v3}/sync`.
|
||||
([!12](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/12))
|
||||
6. **BREAKING:** Allow federation by default.
|
||||
5. **BREAKING:** Allow federation by default.
|
||||
([!24](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/24))
|
||||
* If you relied on federation being disabled by default, make sure to
|
||||
explicitly disable it before upgrading.
|
||||
7. **BREAKING:** Remove the `[global]` section from the configuration file.
|
||||
6. **BREAKING:** Remove the `[global]` section from the configuration file.
|
||||
([!38](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/38))
|
||||
* Details on how to migrate can be found in the merge request's description.
|
||||
8. **BREAKING:** Allow specifying multiple transport listeners in the
|
||||
7. **BREAKING:** Allow specifying multiple transport listeners in the
|
||||
configuration file.
|
||||
([!39](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/39))
|
||||
* Details on how to migrate can be found in the merge request's description.
|
||||
9. Increase default log level so that span information is included.
|
||||
8. Increase default log level so that span information is included.
|
||||
([!50](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/50))
|
||||
10. **BREAKING:** Reorganize config into sections.
|
||||
([!49](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/49))
|
||||
* Details on how to migrate can be found in the merge request's description.
|
||||
11. Try to generate thumbnails for remote media ourselves if the federation
|
||||
9. **BREAKING:** Reorganize config into sections.
|
||||
([!49](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/49))
|
||||
* Details on how to migrate can be found in the merge request's description.
|
||||
10. Try to generate thumbnails for remote media ourselves if the federation
|
||||
thumbnail request fails.
|
||||
([!58](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/58))
|
||||
12. **BREAKING:** Disable unauthenticated access to media by default, set the
|
||||
11. **BREAKING:** Disable unauthenticated access to media by default, set the
|
||||
`serve_media_unauthenticated` config option to `true` to enable it.
|
||||
([!103](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/103))
|
||||
13. **BREAKING:** Split CLI into multiple subcommands. The CLI invocation to run
|
||||
12. **BREAKING:** Split CLI into multiple subcommands. The CLI invocation to run
|
||||
the server is now behind the `serve` command, so `grapevine --config ...`
|
||||
becomes `grapevine serve --config ...`.
|
||||
([!108](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/108))
|
||||
|
|
@ -275,3 +274,5 @@ This will be the first release of Grapevine since it was forked from Conduit
|
|||
24. Add a `check-config` CLI subcommand to check whether the configuration file
|
||||
is valid.
|
||||
([!121](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/121))
|
||||
25. Add configuration options to tune the value of each cache individually.
|
||||
([!124](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/124))
|
||||
|
|
|
|||
|
|
@ -46,11 +46,8 @@ pub(crate) struct Config {
|
|||
pub(crate) database: DatabaseConfig,
|
||||
#[serde(default)]
|
||||
pub(crate) federation: FederationConfig,
|
||||
|
||||
#[serde(default = "default_cache_capacity_modifier")]
|
||||
pub(crate) cache_capacity_modifier: f64,
|
||||
#[serde(default = "default_pdu_cache_capacity")]
|
||||
pub(crate) pdu_cache_capacity: usize,
|
||||
#[serde(default)]
|
||||
pub(crate) cache: CacheConfig,
|
||||
#[serde(default = "default_cleanup_second_interval")]
|
||||
pub(crate) cleanup_second_interval: u32,
|
||||
#[serde(default = "default_max_request_size")]
|
||||
|
|
@ -77,6 +74,38 @@ pub(crate) struct Config {
|
|||
pub(crate) emergency_password: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
#[serde(deny_unknown_fields, default)]
|
||||
pub(crate) struct CacheConfig {
|
||||
pub(crate) pdu: usize,
|
||||
pub(crate) auth_chain: usize,
|
||||
pub(crate) short_eventid: usize,
|
||||
pub(crate) eventid_short: usize,
|
||||
pub(crate) statekey_short: usize,
|
||||
pub(crate) short_statekey: usize,
|
||||
pub(crate) server_visibility: usize,
|
||||
pub(crate) user_visibility: usize,
|
||||
pub(crate) state_info: usize,
|
||||
pub(crate) roomid_spacechunk: usize,
|
||||
}
|
||||
|
||||
impl Default for CacheConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
pdu: 150_000,
|
||||
auth_chain: 100_000,
|
||||
short_eventid: 100_000,
|
||||
eventid_short: 100_000,
|
||||
statekey_short: 100_000,
|
||||
short_statekey: 100_000,
|
||||
server_visibility: 100,
|
||||
user_visibility: 100,
|
||||
state_info: 100,
|
||||
roomid_spacechunk: 200,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub(crate) struct ServerDiscovery {
|
||||
|
|
@ -427,19 +456,11 @@ fn default_db_cache_capacity_mb() -> f64 {
|
|||
300.0
|
||||
}
|
||||
|
||||
fn default_cache_capacity_modifier() -> f64 {
|
||||
1.0
|
||||
}
|
||||
|
||||
#[cfg(feature = "rocksdb")]
|
||||
fn default_rocksdb_max_open_files() -> i32 {
|
||||
1000
|
||||
}
|
||||
|
||||
fn default_pdu_cache_capacity() -> usize {
|
||||
150_000
|
||||
}
|
||||
|
||||
fn default_cleanup_second_interval() -> u32 {
|
||||
// every minute
|
||||
60
|
||||
|
|
|
|||
|
|
@ -67,14 +67,7 @@ impl Services {
|
|||
alias: rooms::alias::Service::new(db),
|
||||
auth_chain: rooms::auth_chain::Service::new(
|
||||
db,
|
||||
#[allow(
|
||||
clippy::as_conversions,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_possible_truncation
|
||||
)]
|
||||
{
|
||||
(100_000.0 * config.cache_capacity_modifier) as usize
|
||||
},
|
||||
config.cache.auth_chain,
|
||||
),
|
||||
directory: db,
|
||||
edus: rooms::edus::Service {
|
||||
|
|
@ -91,89 +84,30 @@ impl Services {
|
|||
search: db,
|
||||
short: rooms::short::Service::new(
|
||||
db,
|
||||
#[allow(
|
||||
clippy::as_conversions,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_possible_truncation
|
||||
)]
|
||||
{
|
||||
(100_000.0 * config.cache_capacity_modifier) as usize
|
||||
},
|
||||
#[allow(
|
||||
clippy::as_conversions,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_possible_truncation
|
||||
)]
|
||||
{
|
||||
(100_000.0 * config.cache_capacity_modifier) as usize
|
||||
},
|
||||
#[allow(
|
||||
clippy::as_conversions,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_possible_truncation
|
||||
)]
|
||||
{
|
||||
(100_000.0 * config.cache_capacity_modifier) as usize
|
||||
},
|
||||
#[allow(
|
||||
clippy::as_conversions,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_possible_truncation
|
||||
)]
|
||||
{
|
||||
(100_000.0 * config.cache_capacity_modifier) as usize
|
||||
},
|
||||
config.cache.short_eventid,
|
||||
config.cache.eventid_short,
|
||||
config.cache.statekey_short,
|
||||
config.cache.short_statekey,
|
||||
),
|
||||
state: rooms::state::Service {
|
||||
db,
|
||||
},
|
||||
state_accessor: rooms::state_accessor::Service::new(
|
||||
db,
|
||||
#[allow(
|
||||
clippy::as_conversions,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_possible_truncation
|
||||
)]
|
||||
{
|
||||
(100.0 * config.cache_capacity_modifier) as usize
|
||||
},
|
||||
#[allow(
|
||||
clippy::as_conversions,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_possible_truncation
|
||||
)]
|
||||
{
|
||||
(100.0 * config.cache_capacity_modifier) as usize
|
||||
},
|
||||
config.cache.server_visibility,
|
||||
config.cache.user_visibility,
|
||||
),
|
||||
state_cache: rooms::state_cache::Service::new(db),
|
||||
state_compressor: rooms::state_compressor::Service::new(
|
||||
db,
|
||||
#[allow(
|
||||
clippy::as_conversions,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_possible_truncation
|
||||
)]
|
||||
{
|
||||
(100.0 * config.cache_capacity_modifier) as usize
|
||||
},
|
||||
),
|
||||
timeline: rooms::timeline::Service::new(
|
||||
db,
|
||||
config.pdu_cache_capacity,
|
||||
config.cache.state_info,
|
||||
),
|
||||
timeline: rooms::timeline::Service::new(db, config.cache.pdu),
|
||||
threads: rooms::threads::Service {
|
||||
db,
|
||||
},
|
||||
spaces: rooms::spaces::Service::new(
|
||||
#[allow(
|
||||
clippy::as_conversions,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_possible_truncation
|
||||
)]
|
||||
{
|
||||
(200.0 * config.cache_capacity_modifier) as usize
|
||||
},
|
||||
config.cache.roomid_spacechunk,
|
||||
),
|
||||
user: db,
|
||||
},
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
---
|
||||
source: tests/integrations/check_config.rs
|
||||
description: A config with invalid keys fails
|
||||
snapshot_kind: text
|
||||
---
|
||||
Error: failed to validate configuration
|
||||
Caused by: failed to parse configuration file "tests/integrations/fixtures/check_config/invalid-keys.toml"
|
||||
|
|
@ -9,4 +8,4 @@ Error: failed to validate configuration
|
|||
|
|
||||
1 | some_name = "example.com"
|
||||
| ^^^^^^^^^
|
||||
unknown field `some_name`, expected one of `conduit_compat`, `listen`, `tls`, `server_name`, `server_discovery`, `database`, `federation`, `cache_capacity_modifier`, `pdu_cache_capacity`, `cleanup_second_interval`, `max_request_size`, `allow_registration`, `registration_token`, `allow_encryption`, `allow_room_creation`, `serve_media_unauthenticated`, `default_room_version`, `proxy`, `jwt_secret`, `observability`, `turn`, `emergency_password`
|
||||
unknown field `some_name`, expected one of `conduit_compat`, `listen`, `tls`, `server_name`, `server_discovery`, `database`, `federation`, `cache`, `cleanup_second_interval`, `max_request_size`, `allow_registration`, `registration_token`, `allow_encryption`, `allow_room_creation`, `serve_media_unauthenticated`, `default_room_version`, `proxy`, `jwt_secret`, `observability`, `turn`, `emergency_password`
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue