.. Licensed under the Apache License, Version 2.0 (the "License"); you may not .. use this file except in compliance with the License. You may obtain a copy of .. the License at .. .. http://www.apache.org/licenses/LICENSE-2.0 .. .. Unless required by applicable law or agreed to in writing, software .. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT .. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the .. License for the specific language governing permissions and limitations under .. the License. .. _release/3.3.x: ============ 3.3.x Branch ============ .. contents:: :depth: 1 :local: .. _release/3.3.3: Version 3.3.3 ============= Features and Enhancements ------------------------- * :ghissue:`4623`: Handle replicator instance start time during upgrades better. * :ghissue:`4653`: Fix the ability to use ``;`` in config values. * :ghissue:`4626`: Fix purge infos replicating to the wrong shards during shard splitting. * :ghissue:`4669`: Make it possible to override ``[replicator] valid_socket_options`` with more than two items. * :ghissue:`4670`: Allow setting of some ibrowse replication options such as ``{prefer_ipv6, true}``. * :ghissue:`4679`: Fix multipart parser "attachment longer than expected" error. Previously there was a small chance attachments were not replicated. * :ghissue:`4680`: Allow restarting failed shard splitting jobs. * :ghissue:`4722`: Fix badmatch error when purge requests time out. * :ghissue:`4736`: Stop client process and clean up if client disconnects when processing streaming requests. * :ghissue:`4758`: Remove sensitive headers from the mochiweb request in process dictionary. This should prevent sensitive headers leaking into logs when a request process crashes. * :ghissue:`4784`: Extract the correct node name from ``ERL_FLAGS`` in ``remsh``. * :ghissue:`4794`: Fix incorrect raising of ``database_does_not_exist`` error. * :ghissue:`4821`: Wait for compacted indexes to flip. Previously, a timeout during compact file flips could lead to crashes and a subsequent recompaction. * :ghissue:`4837`: Fix update bug in ets_lru. * :ghissue:`4847`: Require auth for ``_replicate`` endpoint. * :ghissue:`4878`: Add an option to scrub some sensitive headers from external json requests. .. _release/3.3.2: Version 3.3.2 ============= Features and Enhancements ------------------------- * :ghissue:`4529`: In Javascript process manager, use a database tag in addition to a ddoc ID to quickly find processes. This should improve performance. * :ghissue:`4509`, :ghissue:`4405`: Make remsh work with quoted cookie values. * :ghissue:`4473`: Avoid re-compiling filter view functions. This could speed up Javascript filter functions. * :ghissue:`4412`: Remove Javascript json2 script and the try/except clause around seal. * :ghissue:`4513`: Allow configurable timeouts for ``_view`` and ``_search``. Search timeouts can be specified as ``[fabric] search_timeout`` and ``[fabric] search_permsg``. View per-message timeout can be configured as ``[fabric] view_permsg_timeout``. * :ghissue:`4438`: Proxy auth can now use one of the configured hash algorithms from chttpd_auth/hash_algorithms to decode authentication tokens. * :ghissue:`4370`: Ensure design docs are uploaded individually when replicating with ``_bulk_get``. This restores previous behavior before version 3.3.0. * :ghissue:`4416`: Allow ``_local`` doc writes to the replicator dbs. Previously this issue prevented replicating the replicator db itself, since checkpointing was not working properly. * :ghissue:`4363`: Fix replication ``_scheduler/docs`` ``"total_rows"`` value. * :ghissue:`4380`: Be more defensive about SpiderMonkey location. An error should be emitted early if the Spidermonkey library cannot be found. * :ghissue:`4388`: Bump recon to 2.5.3. See the `changelog `_ for more details. * :ghissue:`4476`, :ghissue:`4515`, :ghissue:`4490`, :ghissue:`4350`, :ghissue:`4379`: Various documentation cleanups and fixes. * Fix for `CVE-2023-26268 `_. .. _release/3.3.1: Version 3.3.1 ============= Features and Enhancements ------------------------- * :ghissue:`4343`, :ghissue:`4344`, :ghissue:`4345`: Fix ``undef`` when parsing replication doc body with a ``user_ctx``. * :ghissue:`4346`: Add ``make`` target to find ``undef`` errors. * :ghissue:`4347`: Remove failed ``couch_plugins`` experiment, fixes more ``undef`` errors. * :ghissue:`4348`: Fix ``undef`` error in ``weatherreport``. * :ghissue:`4353`: Allow starting of more than one replication job. (D’OH!) .. _release/3.3.0: Version 3.3.0 ============= Highlights ---------- * :ghissue:`4308`: Replicator was optimized and should be faster. It now uses the `_bulk_get` endpoint on the source, and can statistically skip calling `_revs_diff` on the target. Benchmark tests replicating 1M documents, 10KB each, from UK to US East show a 3x speed improvement. .. figure:: ../../images/tea-erl-grey-hot-replicator.png :align: center :alt: Replicator, Tea! Earl Grey! Hot! (Because Picard said so) Features and Enhancements ------------------------- * :ghissue:`3766`, :ghissue:`3970`, :ghissue:`3972`, :ghissue:`4093`, :ghissue:`4102`, :ghissue:`4104`, :ghissue:`4110`, :ghissue:`4111`, :ghissue:`4114`, :ghissue:`4245`, :ghissue:`4246`:, :ghissue:`4266`: Add ``smoosh`` queue persistence. This allows resuming ``smoosh`` operations after a node restart. This is disabled by default and can be enabled with ``[smoosh] persist = true``. Optimise ``smoosh`` operations and increase test coverage to 90%. * :ghissue:`3798`: Add ``libicu`` version and collation algorithm version to ``/_node/{node-name}/_versions``. * :ghissue:`3837`: The Erlang source tree is now auto-formatted with ``erlfmt``. * :ghissue:`3845`: Clean up the ``couch_ejson_compare`` C-module and squash Microsoft compiler warnings. * :ghissue:`3832`: Add ``GET`` variant to ``_dbs_info`` endpoint, used to be ``POST`` only. * :ghissue:`3864`: Improve ``erlang_ls`` configuration. * :ghissue:`3853`: Remove legacy ``ddoc_cache_opener`` ``gen_server`` and speed up event routing. * :ghissue:`3879`: Remove use of ``ERL_OPTS`` environment variable. All supported Erlang versions now use ``ERL_COMPILER_OPTIONS`` for the same purpose. * :ghissue:`3883`: Add support for SpiderMonkey 91. * :ghissue:`3889`: Track ``libicu`` collator versions in the view header. * :ghissue:`3952`: Make the timeout for receiving requests from attachment writers configurable. * :ghissue:`3927`: Include index signature in ``_search_info``. * :ghissue:`3963`: Optimtize key tree stemming by using maps instead of sets. This greatly reduced memory usage for heavily conflicted docs in some situations. * :ghissue:`3974`: Create new config options in ``[couchdb]`` and ``[smoosh]`` sections to enable finer control of compaction logging levels. * :ghissue:`3983`, :ghissue:`3984`, :ghissue:`3985`, :ghissue:`3987`, :ghissue:`4033`: Add various functions to ``couch_debug`` module. * :ghissue:`4000`: Ensure ``Object.prototype.toSource()`` is always available. * :ghissue:`4018`: Update ``jiffy`` to 1.1.1 and ``b64url`` to 1.0.3. * :ghissue:`4021`: Reduce smoosh compaction log level to ``debug``. * :ghissue:`4041`: Allow and evaluate nested json claim roles in JWT token. * :ghissue:`4060`, :ghissue:`4290`: Add support for Erlang 25. * :ghissue:`4064`: Enable replicating purge requests between nodes. Also avoid applying interactive purges more than once. * :ghissue:`4069`, :ghissue:`4084`: Drop support for Erlang < 23, update ``vm.args`` settings to match. Review this if you have customized your ``vm.args``. * :ghissue:`4083`: Support Elixir 13. * :ghissue:`4085`: Add an option to let ``custodian`` always use ``[cluster] n`` value. * :ghissue:`4095`: Implement ``winning_revs_only`` option for the replicator. It replicates only the winning revisions from the source to the target, effectively discarding conflicts. * :ghissue:`4135`: Separate search IO from file IO. * :ghissue:`4140`, :ghissue:`4162`: Upgrade hash algorithm for cookie auth (sha1 -> sha256). This introduces a new config setting ``hash_algorithms``. New cookie values are hashed with sha256, sha1 hashes are still accepted. Admins can set this to sha256 only. Sha1 will be disallowed in the next major release. Show supported hash algorithms in ``/_node/{node-name}/_versions`` endpoint. * :ghissue:`4179`: Don't double-encode changes sequence strings in the replicator. * :ghissue:`4182`: Explicitly maintain a fully connected cluster. Previously, it was possible for the nodes to disconnect, and for that state to persist until the nodes restarted. * :ghissue:`4198`: Redact passwords in log file. * :ghissue:`4243`: Update ``mochiweb`` to 3.1.1. * :ghissue:`4254`: The ``_dbs_info`` access control is now configured with the ``[couchdb] admin_only_all_dbs`` setting. Defaults to true. This was a leftover from the 3.0.0 release. * :ghissue:`4264`: ``active`` database sizes is now limited to leaf nodes. Previously, it included intermediate tree nodes, which had the effect that deleting (large) documents did not decrease ``active`` database size. In addition, ``smoosh`` now picks up databases where large documents are deleted for compaction more eagerly, reclaiming the deleted space quicker. * :ghissue:`4270`: Shard splitting now uses its own ``reshard`` IO priority. It can be configured to be safely run in the background with production loads, or with maximum IO available, if admins prefer quicker progress. * :ghissue:`4274`: Improve validation of replicator job parameters & move ``_replicator`` VDU design doc to internal BDU. * :ghissue:`4280`: Add ``CFLAGS`` and ``LDFLAGS`` to ICU build parameters. * :ghissue:`4284`: Remove all usage of global to avoid potential deadlocks in replication jobs. * :ghissue:`4287`: Allow ``=`` in config key names. * :ghissue:`4306`: Fauxton was updated to version v1.2.9. Changes since v1.2.8 can be found `here `_ * :ghissue:`4317`: Write "Relax" welcome message to standard out on Windows. Performance ----------- * :ghissue:`3860`: Add sharding to ``couch_index_server``, similar to :ghissue:`3366`, avoids processing bottlenecks on servers with a lot of concurrent view indexing going on. * :ghissue:`3891`: Avoid decoding JWT payloads when not necessary. * :ghissue:`4031`: Default ``[rexi] use_kill_all`` to ``true``. This improves intra-cluster-node messaging. Set to false if you run a cluster with nodes that have a version <3.0.0. * :ghissue:`4052`: Optimise ``couch_util:reorder_results/2,3``, which speeds up ``_bulk_docs`` and ``_revs_diff``. * :ghissue:`4055`: Avoid using ``length/1`` guard for ``>0`` or ``==0`` tests in ``couch_key_tree``. * :ghissue:`4056`: Optimise ``couch_key_tree:find_missing/2``. This speeds up ``_revs_diff``. * :ghissue:`4059`: Reduce complexity of ``possible_ancestors`` from quadratic to linear. This speeds up working with heavily conflicted documents significantly. * :ghissue:`4091`: Optimise ``couch_util:to_hex/1``, this speeds up all operations that need to encode a revision id into JSON (this is most operations). * :ghissue:`4106`: Set ``io_priority`` in all IO paths. Introduces ``system`` ``io_priority``. * :ghissue:`4144`, :ghissue:`4172`: Implement ``_bulk_get`` support for the replicator. Backward compatibility is ensured. This speeds up all replications. Add option to disable new behaviour for legacy setups. * :ghissue:`4163`: Statistically skip ``_revs_diff`` in the replicator. This improves performance for replications into empty targets. * :ghissue:`4177`: Remove the long deprecated ``bigcouch 0.4`` change sequence support. * :ghissue:`4238`: Optimise ``_bulk_get`` endpoint. This speeds up replication of 1M docs by ~2x. Individual ``_bulk_get`` requests are up to 8x faster. * :ghissue:`3517`: Add experimental fix for reduce performance regression due to expensive repeated AST-transformations on newer SpiderMonkey versions. Set ``COUCHDB_QUERY_SERVER_JAVASCRIPT`` env var to ``COUCHDB_QUERY_SERVER_JAVASCRIPT="/opt/couchdb/bin/couchjs /opt/couchdb/share/server/main-ast-bypass.js"``. * :ghissue:`4262`: ``couchjs`` executable built against Spidermonkey >= 78 will return the detailed ``major.minor.patch`` as opposed to just the ``major`` version as previously. Bugfixes -------- * :ghissue:`3817`: Fix undefined function call in ``weatherreport``. * :ghissue:`3819`: Return ``400`` instead of ``500`` response code for known invalid ``_bulk_docs`` with ``new_edits=false`` request. * :ghissue:`3861`: Add ``SameSite`` setting when clearing session cookies. * :ghissue:`3863`: Fix custom TLS distribution for Erlang 20. * :ghissue:`3870`: Always send all cookie attributes. * :ghissue:`3886`: Avoid changes feed rewind after shard move with no subsequent db updates. * :ghissue:`3888`: Make ``_stats`` endpoint resilient against nodes that go offline. * :ghissue:`3901`: Use db-creation time instead of ``0`` for ``instance_start_time`` to help replicator recognise whether a peer database was deleted and recreated. * :ghissue:`3909`: Fix ``new_edits:false`` and VDU ``function_clause``. * :ghissue:`3934`: Fix ``replicated_changes`` typo for purge doc updates. * :ghissue:`3940`: Ensure the multipart parser always monitors the worker and make sure to wait for attachment uploads before responding. * :ghissue:`3950`: Ignore responses from timed-out or retried ``ibrowse`` calls. * :ghissue:`3969`: Fix ``skip`` and ``limit`` for ``_all_dbs`` and ``_dbs_info``. * :ghissue:`3979`: Correctly respond with a ``500`` code when document updates time out under heavy load. * :ghissue:`3992`: Show that Search is available if it was available before. Avoid Search availability disappearing just because a Search node was temporarily not available. * :ghissue:`3993`: Return a ``400`` error when decoding a JWT token fails, rather than crashing and not responding at all. * :ghissue:`3990`: Prevent creation of ddocs with no name through Mango index creation. * :ghissue:`4003`: Improve index building during shard splitting. * :ghissue:`4016`: Fix ``function_clause`` error for replicated changes with a target VDU. * :ghissue:`4020`: Fix ``maybe_handle_error`` clauses. * :ghissue:`4037`: Fix ES{256,384,512} support for JWTs. * :ghissue:`4040`: Handle ``exit(shutdown)`` error in ``chttpd``. * :ghissue:`4043`: Fix purge request timeouts (5s -> infinity). * :ghissue:`4146`: The ``devcontainer`` has been updated. * :ghissue:`4050`: Handle ``all_dbs_active`` in ``fabric_doc_update``. * :ghissue:`4160`: Return a proper ``400`` error when an invalid object is sent to ``_bulk_get``. * :ghissue:`4070`: Prevent ``error:function_clause`` in ``check_security/3`` if roles claim is malformed. * :ghissue:`4075`: Fix ``couch_debug:opened_files*`` functions. * :ghissue:`4108`: Trim ``X-Auth-CouchDB-Roles`` header after reading. * :ghissue:`4153`: The ``require_valid_user`` setting is now under ``chttpd``. * :ghissue:`4161`: Fix ``content-type`` handling in ``_session``. * :ghissue:`4176`: Fix ``eventsource`` ``_changes`` feed. * :ghissue:`4197`: Support large (and impractical as-of-yet) ``q`` values. Fix shard open timeouts for ``q > 64``. * :ghissue:`4199`: Fix spurious unlock in ``close_db_if_idle``. * :ghissue:`4230`: Avoid refresh messages piling up in prometheus server. * :ghissue:`4240`: Implement global password hasher process. This fixes a race-condition when setting new admin passwords in quick succession on a multicore server. * :ghissue:`4261`, :ghissue:`4271`: Clean up stale view checkpoints, improve purge client cleanup logging * :ghissue:`4272`: Kill all ``couch_server_N`` if ``database_dir`` changes. * :ghissue:`4313`: Use ``chttpd`` config section when patching local ``_replicate`` endpoints. * :ghissue:`4321`: Downgrade jiffy to allow building on Windows again. * :ghissue:`4329`, :ghissue:`4323`: Ignore build windows binaries in git. Tests ----- * :ghissue:`3825`: Eliminate Elixir compiler warnings. * :ghissue:`3830`: Reduce skipped Elixir integration tests. * :ghissue:`3890`: Handle ``not_found`` lookups removing ddoc cache key. * :ghissue:`3892`: Use Debian Stable for CI, add Erlang 24 to CI. * :ghissue:`3898`: Remove CI support for Ubuntu 16.04. * :ghissue:`3903`, :ghissue:`3914`: Refactor Jenkins to dynamically generate stages. Drop ``MINIMUM_ERLANG_VERSION`` to 20, drop the packaging ``ERLANG_VERSION`` to 23, add the ``weatherreport-test`` as a build step, and add ``ARM`` and ``POWER`` back into the matrix. * :ghissue:`3921`:, :ghissue:`3923`: Execute various tests in clean ``database_dir`` to avoid subsequent test flakiness. * :ghissue:`3968`: Ensure key tree rev stemming does’t take too much memory. * :ghissue:`3980`: Upgrade Mango test dependency ``nose`` to ``nose`` and fix flaky-on-Windows tests. * :ghissue:`4006`: Remove CI support for Debian 9. * :ghissue:`4061`, :ghissue:`4082`: Update PPC CI builder. * :ghissue:`4096`: Fix flaky ``validate_doc_update`` Elixir test. * :ghissue:`4123`: Fix ``haproxy.cfg``. * :ghissue:`4126`: Return a ``400`` response for a single ``new_edits=false`` doc update without revision. * :ghissue:`4129`: Fix ``proxyauth_test`` and removed it from skip list. * :ghissue:`4132`: Address race condition in ``cpse_incref_decref`` test. * :ghissue:`4151`: Refactor replication tests to use clustered endpoints. * :ghissue:`4178`: Add test coverage to prevent junk in ``eventsource``. * :ghissue:`4188`: Enable eunit coverage for all applications instead of enabling it per-application. * :ghissue:`4202`: Fix race condition in ddoc cache LRU test. * :ghissue:`4203`, :ghissue:`4205`: Reduce test log noise. * :ghissue:`4268`: Improve flaky ``_dbs_info`` test. * :ghissue:`4319`: Fix offline ``configure`` and ``make release``. * :ghissue:`4328`: Fix ``eaddrnotavail`` in Elixir tests under Windows. * :ghissue:`4330`: Do not run source checks in main CI build. Docs ---- * :ghissue:`4164`: The CouchDB documentation has been moved into the main CouchDB repository. * :ghissue:`4307`, :ghissue:`4174`: Update Sphinx to version 5.3.0 * :ghissue:`4170`: Document the ``/_node/{node-name}/_versions`` endpoint. Builds ------ * :ghissue:`4097`: Stop publication of nightly packages. They were not used anywhere. * :ghissue:`4322`: Reuse installed rebar and rebar3 for mix. Compatible with Elixir =< 13 only. Elixir 14 is not supported yet. * :ghissue:`4326`: Move Elixir source checks to a separate build step. Other ----- * Added pumpkin spice to selected endpoints. — Thank you for reading the 3.3.0 release notes.