1.8. 2.1.x Branch¶
1.8.1. Upgrade Notes¶
When upgrading from 2.x to 2.1.1, if you have not customized your node name in
vm.args
, be sure to retain your originalvm.args
file. The default node name has changed fromcouchdb@localhost
tocouchdb@127.0.0.1
, which can prevent CouchDB from accessing existing databases on the system. You may also change the name option back to the old value by setting-name couchdb@localhost
inetc/vm.args
by hand. The default has changed to meet new guidelines and to provide additional functionality in the future.If you receive errors in the logfile, such as
internal_server_error : No DB shards could be opened.
or in Fauxton, such asThis database failed to load.
you need to make this change.The deprecated (and broken) OAuth 1.0 implementation has been removed.
If user code reads or manipulates replicator document states, consider using the
[replicator] update_docs = true
compatibility parameter. In that case the replicator will continue updating documents with transient replication states. However, that will incur a performance cost. Consider instead using the_scheduler/docs
HTTP endpoint.The
stale
parameter for views and_find
has been deprecated in favour of two new parameters:stable
andupdate
. The oldstale=ok
behaviour is equivalent tostable=true&update=false
, and the oldstale=update_after
behaviour is equivalent tostable=true&update=lazy
. The deprecatedstale
parameter will be removed in CouchDB 3.0.The new :
httpd/max_http_request_size
configuration parameter was added. This has the same behavior as the oldcouchdb/max_document_size
configuration parameter, which had been unfortunately misnamed, and has now been updated to behave as the name would suggest. Both are documented in the shippeddefault.ini
file.Note that the default for this new parameter is 64MB instead of 4GB. If you get errors when trying to PUT or POST and see HTTP 413 return codes in couchdb logs, this could be the culprit. This can affect couchup in-place upgrades as well.
#914: Certain critical config sections are blacklisted from being modified through the HTTP API. These sections can still be modified through the standard
local.ini
orlocal.d/*.ini
files.#916:
couchjs
now disableseval()
and theFunction()
constructor by default. To restore the original behaviour, add the--eval
flag to the definition of the javascript query server in yourlocal.ini
file.
1.8.2. Version 2.1.2¶
1.8.2.1. Security¶
1.8.3. Version 2.1.1¶
1.8.3.1. Security¶
1.8.3.2. General¶
#617: CouchDB now supports compilation and running under Erlang/OTP 20.x.
#756: The
couch_peruser
functionality is now really fixed. Really.#827: The cookie domain for AuthSession cookies, used in a proxy authentication configuration, can now be customized via the ini file.
#858: It is now possible to modify shard maps for system databases.
#732: Due to an Erlang bug (ERL-343), invalid paths can be returned if volumes are mounted containing whitespace in their name. This problem surfaced primarily on macOS (Time Machine volumes). CouchDB now works around this bug in unpatched versions of Erlang by skipping the free space check performed by the compaction daemon. Erlang itself will correctly perform free space checks in version 21.0.
#824: The current node’s local interface can now be accessed at
/_node/_local/{endpoint}
as well as at/_node/<nodename>@<hostname>/{endpoint}
.The Dockerfile in the source repository has been retired. For a current Dockerfile, see the couchdb-docker repository.
Fauxton now uses a version of React with a BSD license.
1.8.3.3. Performance¶
#835: CouchDB now no longer decompresses documents just to determine their uncompressed size. In tests, this has lead to improvements between 10-40% in both CPU and wall-clock time for database compaction.
The design document cache (
ddoc_cache
) has been rewritten to improve performance.
1.8.3.4. Mango¶
#808: Mango now supports partial indexes. Partial indexes allow documents to be filtered at indexing time, potentially offering significant performance improvements for query selectors that don’t map cleanly to a range query on an index.
#740: Mango queries can now be paginated. Each query response includes a bookmark. The bookmark can be provided on a subsequent query to continue from a specific key.
#768: Mango
_find
accepts anexecution_stats
parameter. If present, a new object is included in the response which contains information about the query executed. The object contains the count of total keys examined (0 for json indexes), total documents examined (wheninclude_docs=true
is used), and the total quorum documents examined (when fabric doc lookups are used).#816 and #866: Mango now requires that all of the fields in a candidate index must exist in a query’s selector. Previously, this check was incorrect, and indexes that might only contain a subset of valid documents might be selected by the query planner if no explicit index was specified at query time. Further, if a sort field is specified at query time, that field needs to exist (but could be null) in the results returned.
1.8.3.5. Other¶
The 2.1.1 release also includes the following minor improvements:
#635: Stop couch_index processes on ddoc update
#721: Save migrated replicator checkpoint documents immediately
#688: Reuse http-based replication checkpoints when upgrading to https
#729: Recommend the use only of
-name
and not-sname
in vm.args for compatibility.#738: Allow replicator application to always update replicator docs.
#605: Add
Prefer: return=minimal
header options from RFC7240 to reduce the number of headers in the response.#744: Allow a 503 response to be returned to clients (with metric support)
#746: Log additional information on crashes from rexi
#752: Allow Mango $in queries without requiring the index to use an array
(multiple) Additional debugging utilities have been added.
(multiple) Hot code upgrades from 2.0 -> 2.1.1 are now possible.
(multiple) Improvements to the test suite have been made.
#765: Mango
_explain
now includes view parameters as requested by the user.#653: _show and _list should now work for admin-only databases such as
_users
.#807: Mango index selection should occur only once.
#804: Unhandled Mango errors are now logged.
#659: Improve accuracy of the
max_document_size
check.#817: Invalid Base64 in inline attachments is now caught.
#825: Replication IDs no longer need to be URL encoded when using the
_scheduler/jobs/<job_id>
endpoint.#838: Do not buffer rexi messages to disconnected nodes.
#830: The stats collection interval is now configurable in an ini file, not in the application context. The default value is 10, and the setting is reloaded every 600 seconds.
#812: The
/{db}
endpoint now includes acluster
block with the database’sq
,n
, and defaultw
andr
values. This supplements the existing/{db}/_shards
and/{db}/_shards/{id}
detailed information on sharding and quorum.#810: The replicator scheduler crashed counter gauge more reliably detects replication crashes by reducing the default number of retries from 10 to 5 (reducing the duration from 4 mins to 8 secs).
COUCHDB-3288: Tolerate mixed clusters for the upcoming pluggable storage engine work.
#839: Mango python tests now support Python 3 as well as 2.
#845: A convenience
remsh
script has been added to support live debugging of running systems.#846: Replicator logging is now less verbose and more informative when replication terminates unexpectedly.
#797: Reduce overflow errors are now returned to the client, allowing views with a single bad reduce to build while not exhausting the server’s RAM usage.
#881: Mango now allows match on documents where the indexed value is an object if a range query is issued. Previously, query results might change in the presence of an index, and operators/selectors which explicitly depend on a full index scan (such as
$exists
) would not return a complete result set.#883: Erlang time module compatibility has been improved for releases of Erlang newer than 18.0.
#933: 410 is now returned when attempting to make a temporary view request.
#934: The replicator now has a configurable delay before retrying to retrieve a document after receiving a
missing_doc
error.#936: jiffy now deduplicates JSON keys.
1.8.4. Version 2.1.0¶
The Mango
_find
endpoint supports a new combination operator,$allMatch
, which matches and returns all documents that contain an array field with all its elements matching all the specified query criteria.New scheduling replicator. The core of the new replicator is a scheduler which allows running a large number of replication jobs by switching between them, stopping some and starting others periodically. Jobs which fail are backed off exponentially. There is also an improved inspection and querying API:
_scheduler/jobs
and_scheduler/docs
:_scheduler/jobs
: This endpoint shows active replication jobs. These are jobs managed by the scheduler. Some of them might be running, some might be waiting to run, or backed off (penalized) because they crashed too many times. Semantically this is somewhat equivalent to_active_tasks
but focuses only on replications. Jobs which have completed or which were never created because of malformed replication documents will not be shown here as they are not managed by the scheduler._replicate
replications, started form _replicate endpoint not from a document in a_replicator
db, will also show up here._scheduler/docs
: This endpoint is an improvement on having to go back and read replication documents to query their state. It represents the state of all the replications started from documents in _replicator db. Unlike_scheduler/jobs
it will also show jobs which have failed or have completed.
By default, scheduling replicator will not update documents with transient states like
triggered
orerror
anymore, instead_scheduler/docs
API should be used to query replication document states.
1.8.4.1. Other scheduling replicator improvements¶
Network resource usage and performance was improved by implementing a shared connection pool. This should help in cases of a large number of connections to the same sources or target. Previously connection pools were shared only within a single replication job.
Improved request rate limit handling. Replicator requests will auto-discover rate limit capacity on targets and sources based on a proven Additive Increase / Multiplicative Decrease feedback control algorithm.
Improved performance by having exponential backoff for all replication jobs failures. Previously there were some scenarios were failure led to continuous repeated retries, consuming CPU and disk resources in the process.
Improved recovery from long but temporary network failure. Currently if replications jobs fail to start 10 times in a row, they will not be retried anymore. This is sometimes desirable, but in some cases, for example, after a sustained DNS failure which eventually recovers, replications reach their retry limit, stop retrying and never recover. Previously it required user intervention to continue. Scheduling replicator will never give up retrying a valid scheduled replication job and so it should recover automatically.
Better handling of filtered replications. Failing user filter code fetches from the source will not block replicator manager and stall other replications. Failing filter fetches will also be backed off exponentially. Another improvement is when filter code changes on the source, a running replication will detect that and restart itself with a new replication ID automatically.
The 2.1.0 release also includes the following minor improvements:
COUCHDB-1946: Hibernate couch_stream after each write (up to 70% reduction in memory usage during replication of DBs with large attachments)
COUCHDB-2964: Investigate switching replicator manager change feeds to using “normal” instead of “longpoll”
COUCHDB-2988: (mango) Allow query selector as changes and replication filter
COUCHDB-2992: Add additional support for document size
COUCHDB-3046: Improve reduce function overflow protection
COUCHDB-3061: Use vectored reads to search for buried headers in .couch files. “On a modern linux system with SSD, we see improvements up to 15x.”
COUCHDB-3063: “stale=ok” option replaced with new “stable” and “update” options.
COUCHDB-3180: Add features list in the welcome message
COUCHDB-3203: Make auth handlers configurable (in ini files)
COUCHDB-3234: Track open shard timeouts with a counter instead of logging
COUCHDB-3242: Make get view group info timeout in couch_indexer configurable
COUCHDB-3249: Add config to disable index all fields (text indexes)
COUCHDB-3251: Remove hot loop usage of filename:rootname/1
COUCHDB-3284: 8Kb read-ahead in couch_file causes extra IO and binary memory usage
COUCHDB-3298: Optimize writing btree nodes
COUCHDB-3302: (Improve) Attachment replication over low bandwidth network connections
COUCHDB-3307: Limit calls to maybe_add_sys_db_callbacks to once per db open
COUCHDB-3318: bypass couch_httpd_vhost if there are none
COUCHDB-3323: Idle dbs cause excessive overhead
COUCHDB-3324: Introduce couch_replicator_scheduler
COUCHDB-3337: End-point _local_docs doesn’t conform to query params of _all_docs
COUCHDB-3358: (mango) Use efficient set storage for field names
COUCHDB-3425: Make _doc_ids _changes filter fast-path limit configurable
#457: TeX/LaTeX/texinfo removed from default docs build chain
#469: (mango) Choose index based on fields match
#483: couchup database migration tool
#582: Add X-Frame-Options support to help protect against clickjacking
#593: Allow bind address of 127.0.0.1 in
_cluster_setup
for single nodes#624: Enable compaction daemon by default
#626: Allow enable node decom using string “true”
(mango) Configurable default limit, defaults to 25.
(mango) _design documents ignored when querying _all_docs
(mango) add $allMatch selector
Add local.d/default.d directories by default and document
Improved INSTALL.* text files
1.8.5. Fixed Issues¶
The 2.1.0 release includes fixes for the following issues:
COUCHDB-1447: X-Couch-Update-NewRev header is missed if custom headers are specified in response of _update handler (missed in 2.0 merge)
COUCHDB-2731: Authentication DB was not considered a system DB
COUCHDB-3010: (Superseded fix for replication exponential backoff)
COUCHDB-3090: Error when handling empty “Access-Control-Request-Headers” header
COUCHDB-3100: Fix documentation on require_valid_user
COUCHDB-3109: 500 when include_docs=true for linked documents
COUCHDB-3113: fabric:open_revs can return {ok, []}
COUCHDB-3149: Exception written to the log if db deleted while there is a change feed running
COUCHDB-3150: Update all shards with stale=update_after
COUCHDB-3158: Fix a crash when connection closes for _update
COUCHDB-3162: Default ssl settings cause a crash
COUCHDB-3164: Request fails when using _changes?feed=eventsource&heartbeat=30000
COUCHDB-3168: Replicator doesn’t handle well writing documents to a target db which has a small max_document_size
COUCHDB-3173: Views return corrupt data for text fields containing non-BMP characters
COUCHDB-3174: max_document_size setting can by bypassed by issuing multipart/related requests
COUCHDB-3178: Fabric does not send message when filtering lots of documents
COUCHDB-3181: function_clause error when adding attachment to doc in _users db
COUCHDB-3184: couch_mrview_compactor:recompact/1 does not handle errors in spawned process
COUCHDB-3193: fabric:open_revs returns multiple results when one of the shards has stem_interactive_updates=false
COUCHDB-3199: Replicator VDU function doesn’t account for an already malformed document in replicator db
COUCHDB-3202: (mango) do not allow empty field names
COUCHDB-3220: Handle timeout in _revs_diff
COUCHDB-3222: (Fix) HTTP code 500 instead of 400 for invalid key during document creation
COUCHDB-3231: Allow fixing users’ documents (type and roles)
COUCHDB-3232: user context not passed down in fabric_view_all_docs
COUCHDB-3238: os_process_limit documentation wrong
COUCHDB-3241: race condition in couch_server if delete msg for a db is received before open_result msg
COUCHDB-3245: Make couchjs -S option take effect again
COUCHDB-3252: Include main-coffee.js in release artifact (broken CoffeeScript view server)
COUCHDB-3255: Conflicts introduced by recreating docs with attachments
COUCHDB-3259: Don’t trap exits in couch_file
COUCHDB-3264: POST to _all_docs does not respect conflicts=true
COUCHDB-3269: view response can ‘hang’ with filter and limit specified
COUCHDB-3271: Replications crash with ‘kaboom’ exit
COUCHDB-3274: eof in couch_file can be incorrect after error
COUCHDB-3277: Replication manager crashes when it finds _replicator db shards which are not part of a mem3 db
COUCHDB-3286: Validation function throwing unexpected json crashes with function_clause
COUCHDB-3289: handle error clause when calling fabric:open_revs
COUCHDB-3291: Excessively long document IDs prevent replicator from making progress
COUCHDB-3293: Allow limiting length of document ID (for CouchDB proper)
COUCHDB-3305: (mango) don’t crash with invalid input to built in reducer function
COUCHDB-3362: DELETE attachment on non-existing document creates the document, rather than returning 404
COUCHDB-3364: Don’t crash compactor when compacting process fails.
COUCHDB-3367: Require server admin user for db/_compact and db_view_cleanup endpoints
COUCHDB-3376: Fix mem3_shards under load
COUCHDB-3378: Fix mango full text detection
COUCHDB-3379: Fix couch_auth_cache reinitialization logic
COUCHDB-3400: Notify couch_index_processes on all shards when ddoc updated
COUCHDB-3402: race condition in mem3 startup
#511: (mango) Return false for empty list
#595: Return 409 to PUT attachment with non-existent rev
#623: Ensure replicator _active_tasks entry reports recent pending changes value
#627: Pass UserCtx to fabric’s all_docs from mango query
#631: fix couchdb_os_proc_pool eunit timeouts
#644: Make couch_event_sup:stop/1 synchronous
#645: Pass db open options to fabric_view_map for _view and _list queries on _users DB
#648: Fix couch_replicator_changes_reader:process_change
#649: Avoid a race when restarting an index updater
#667: Prevent a terrible race condition
#677: Make replication filter fetch error for _replicate return a 404
Fix CORS
max_age
configuration parameter via Access-Control-Max-AgeChunk missing revisions before attempting to save on target (improves replication for very conflicted, very deep revision tree documents)
Allow w parameter for attachments
Return “Bad Request” when count in
/_uuids
exceeds maxFix crashes when replicator db is deleted
Skip internal replication if changes already replicated
Fix encoding issues on
_update/../doc_id
and PUT attachments