Alauda Cache Service E1

Set Up Disaster Recovery Replication

Disaster Recovery (spec.activeRedis.mode: peerof) is the hot-standby topology: an upstream instance fans out directed replication links to one or more downstream instances, each ready to be promoted if the upstream datacenter is lost. Setting it up involves two parts — enabling cross-datacenter replication on each Redis instance, and establishing a connection from the downstream instance to the upstream instance.

For the alternative topology, in which every datacenter accepts writes, see Set Up Active-Active Replication.

Cluster Role Description

In a disaster recovery group, the role of each Redis instance is not fixed; it can be either an upstream or a downstream. The downstream does not restrict data writing, but data written to the downstream will be overwritten by data synchronized from the upstream or cause synchronization failure due to data type conflicts.

TOC

Choose the procedure for your Redis versionRedis 7.2 — new moduleStep 1: Confirm the default account in each datacenterStep 2: Enable replication on the upstream instanceStep 3: Expose the upstream proxyStep 4: Enable replication on the downstream instanceStep 5: Create the connection on the downstream sideStep 6: VerifyRedis 6.0 — legacy moduleUpstream sideUse LoadBalancer as the access address for the upstream ProxyDownstream sidePre-flight inspectionRemoving a connectionCredential rotation

Choose the procedure for your Redis version

The setup procedure differs between the two module generations. Follow the section that matches your Redis version.

Redis versionProcedureLink authentication
7.2Redis 7.2 — new moduleThe instance's default account, bound explicitly
6.0Redis 6.0 — legacy moduleThe instance's default account, bound automatically
Both ends must run the same Redis version

Replication between Redis 6.0 and Redis 7.2 is not supported. See Module generations and version applicability.

Redis 7.2 — new module

The new module requires a peer-auth credential: every instance in the replication group binds a RedisUser through spec.activeRedis.redisUserRef. The operator pushes that credential to every node, and both inbound and outbound peer links authenticate with it. Enabling replication without a binding is rejected at admission, and the module rejects all inbound peer replication.

Bind the instance's own default account. Every instance already has one — its own reconcile creates it — so nothing has to be provisioned; the binding only names it.

Every datacenter needs the same default password

The module accepts an inbound peer only when the credential it presents matches the local peer-auth credential. Binding the default account therefore requires every member of the group to carry the same default password.

This is not automatic: default passwords are generated per instance. Set the same default password on every member before you wire the links.

Step 1: Confirm the default account in each datacenter

The default-account RedisUser is named after the instance:

Instance architectureRedisUser name
Clusterdrc-acl-<instance-name>-default
Sentinel, standalonerfr-acl-<instance-name>-default

Check that it is provisioned in both datacenters — the binding is rejected unless its phase is Success:

$ kubectl -n default get redisusers rfr-acl-s72-dc1-default
NAME                      INSTANCE   USERNAME   PHASE     AGE
rfr-acl-s72-dc1-default   s72-dc1    default    Success   3m

The binding itself is set in Step 2, along with the rest of the replication configuration.

A dedicated account may be bound instead

Binding the default account widens what that password grants: anything holding it can act as a replication peer. A custom RedisUser narrows that, and lets the replication credential be rotated independently of the instance password. It must reference the local instance, carry a password Secret of its own that no other peer-auth RedisUser uses, reach status.phase: Success, and carry the same username and the same password value in every datacenter. A system account is always refused.

Step 2: Enable replication on the upstream instance

$ kubectl -n default patch redis s72-dc1 --type=merge \
    --patch='{"spec": {"activeRedis": {"serviceID": 0, "mode": "peerof", "redisUserRef": "rfr-acl-s72-dc1-default"}}}'
WARNING

The range of serviceID is [0-15] and it must be unique within the replication group. It cannot be changed later, and replication cannot be turned off once enabled.

The instance must also satisfy the new module's constraints, which are enforced by admission:

  • customConfig.appendonly must not be yes — the new module is RDB-only.
  • customConfig.databases must be <= 16 — the module refuses to load beyond 16 databases, and the pods would crash-loop.

Step 3: Expose the upstream proxy

After enabling replication, the proxy provides a NodePort access address by default. A single NodePort-based access address is not highly available. In a production environment, use a LoadBalancer address:

$ kubectl -n default patch redis s72-dc1 --type=merge \
    --patch='{"spec": {"activeRedis": {"proxy": {"service": {"type": "LoadBalancer"}}}}}'

Each instance creates a proxy Service named activeredis-proxy-<instance-name>.

For Sentinel instances, the instance name needs to be prefixed with rfr-.

$ kubectl -n default get svc activeredis-proxy-rfr-s72-dc1
NAME                            TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)          AGE
activeredis-proxy-rfr-s72-dc1   LoadBalancer   10.96.145.32    192.168.1.10    6379:31234/TCP   45s
One endpoint has to be reachable

The proxy Service of a peerof instance exposes a single port, 6379. The proxy carries the RESP control plane and the replication stream on it, telling the two apart per session, so the downstream needs to reach only that one address — the one you put in spec.addresses. No 7379 port is created in Disaster Recovery mode, and none has to be opened between datacenters.

Connection admission dials the endpoint it resolves for the replication stream and rejects the connection if it is unreachable. With spec.peerPort left unset, that is the address in spec.addresses[0] itself.

Step 4: Enable replication on the downstream instance

Confirm the downstream's default account as in Step 1, and make sure its default password matches the upstream's. Then enable replication with a different serviceID:

$ kubectl -n default patch redis s72-dc2 --type=merge \
    --patch='{"spec": {"activeRedis": {"serviceID": 1, "mode": "peerof", "redisUserRef": "rfr-acl-s72-dc2-default"}}}'

Step 5: Create the connection on the downstream side

The ActiveRedisConnection is created in the downstream cluster. spec.instance names the local (downstream) instance, and spec.addresses points at the upstream proxy's RESP endpoint.

$ cat << EOF | kubectl -n default create -f -
apiVersion: redis.middleware.alauda.io/v1alpha1
kind: ActiveRedisConnection
metadata:
  name: conn-dc2-to-dc1
spec:
  instance: s72-dc2
  addresses:
  - 192.168.1.10:6379
  teardownPolicy: Detach
EOF
FieldDescription
instanceThe local (downstream) instance name.
addressesThe upstream proxy's RESP endpoint. The first entry is what the replication stream dials.
peerPortLeave it unset. The upstream proxy serves the replication stream and the RESP control plane on one endpoint, so the port of addresses[0] is used. Set it only when an external load balancer splits the two onto different numbers. There is deliberately no schema default.
teardownPolicyDetach (default) or Decommission. See Removing a connection.
Connection names are used verbatim as module peer names

The connection name becomes the module's peer name, which is stricter than Kubernetes naming:

  • it must start with a letter — 1conn is rejected;
  • reset, clear, all, and list are reserved words and are rejected.
One connection per instance

An instance may hold at most one ActiveRedisConnection — its own upstream link. A one-upstream-to-many-downstreams fan-out is built by creating a connection in each downstream cluster, all pointing at the same upstream.

Step 6: Verify

$ kubectl -n default get activeredisconnections conn-dc2-to-dc1
NAME              INSTANCE   STATUS    MESSAGE   AGE
conn-dc2-to-dc1   s72-dc2    Healthy             40s

$ kubectl -n default get activeredis
NAME                 INSTANCE   SERVICEID   SHARDS   PEERS   DPEERS   PHASE     MESSAGE   AGE
s72-dc2-activeredis  s72-dc2    1           1        1       0        Healthy             3m

The upstream instance's ActiveRedis resource reports the fan-out through status.downstreamPeerCount.

Inspect the connection for per-shard synchronization detail:

$ kubectl -n default get activeredisconnections conn-dc2-to-dc1 -o yaml
...
status:
  instance: s72-dc2
  credentialMode: peer-auth-global
  shards:
  - index: 0
    offset: "15320"
    opId: "412"
    status: Connected
    syncStatus: PartialSync
  status: Healthy
  upstreamPeer:
    service_id: 0
    service_metadata:
      instance: default/s72-dc1

status.shards[].status indicates the connection status of the shard, and status.shards[].syncStatus indicates the data synchronization status: PartialSync (incremental synchronization of Oplog) or FullSync (RDB is being synchronized).

status.credentialMode: peer-auth-global records that the module-side peer records hold no inline credential — outbound dials read the node-local peer-auth credential, so a rotation is picked up on the next re-dial.

The native Redis Sentinel mode does not have the concept of shards. Here, a master-replica pair of Sentinel is abstracted as shard 0 to be compatible with the same data structure as the Redis cluster mode.

Redis 6.0 — legacy module

Legacy implementation

Redis 6.0 carries the frozen legacy module, kept for compatibility with existing instances. It supports Disaster Recovery only — no Active-Active mode, and no module-level peer authentication. Enabling replication on Redis 6.0 returns an admission warning recommending Redis 7.2. For new deployments, use Redis 7.2.

Upstream side

You need to create a Redis instance first.

CLI
Web Console

Enable Disaster Recovery

# Enable ActiveRedis
$ kubectl -n default patch redis s6 --type=merge --patch='{"spec": {"activeRedis":{"serviceID":0 }}}'

Use LoadBalancer as the access address for the upstream Proxy

After enabling disaster recovery support for the instance, the disaster recovery Proxy provides a NodePort access address by default. A single NodePort-based access address is not highly available. In a production environment, you can use a LoadBalancer address to provide access to the Proxy.

CLI
# Enable ActiveRedis with LoadBalancer as Proxy access type
$ kubectl -n default patch redis s6 --type=merge --patch='{"spec": {"activeRedis":{"proxy": {"service": {"type": "LoadBalancer"}}}}}'

Each disaster recovery instance will create a Proxy Service with a name that follows this format: activeredis-proxy-<instance-name>.

For Sentinel instances, the instance name needs to be prefixed with rfr-

Downstream side

CLI
Web Console

Enable Disaster Recovery

# Enable ActiveRedis
$ kubectl -n default patch redis s6-dest --type=merge --patch='{"spec": {"activeRedis":{"serviceID":1 }}}'
WARNING

The range of serviceID is [0-15]. In the same disaster recovery cluster, the serviceID cannot be repeated.

Wait for the peer-auth binding

The operator adds the binding on a following reconcile, and the connection is rejected while it is still empty. Wait until this prints a name:

$ kubectl -n default get redis s6-dest -o jsonpath='{.spec.activeRedis.redisUserRef}'
rfr-acl-s6-dest-default

Configure Disaster Recovery Connection

# create activeredisconnections
$ cat << EOF | kubectl -n default create -f -
apiVersion: redis.middleware.alauda.io/v1alpha1
kind: ActiveRedisConnection
metadata:
name: conn-s6-dest-to-s6-src
spec:
addresses:
- 192.168.1.10:30010
instance: s6-dest
EOF

Note to replace the upstream address.

The link credential is bound on the instance, not on the connection

A Redis 6.0 link authenticates as the instance's default account. The operator writes that down for you: the first time it reconciles an instance that has replication enabled and carries no binding, it sets spec.activeRedis.redisUserRef to the instance's own default-account RedisUser — drc-acl-<instance-name>-default for a cluster instance, rfr-acl-<instance-name>-default for a Sentinel one. Nothing is provisioned: that account already exists, and the password in it is the one the link uses. Both datacenters must therefore carry the same default password.

There is no credential field on the connection. spec.secretName has been removed, and a manifest that still sets it is rejected with unknown field "spec.secretName".

Check Disaster Recovery Connection Status

$ kubectl -n default get activeredisconnections conn-s6-dest-to-s6-src -o yaml
apiVersion: redis.middleware.alauda.io/v1alpha1
kind: ActiveRedisConnection
metadata:
annotations:
  cpaas.io/creator: admin
  cpaas.io/updated-at: "2025-08-26T10:16:49Z"
creationTimestamp: "2025-08-26T10:16:49Z"
generation: 1
labels:
  cpaas.io/activeredis: s6-dest-activeredis
  cpaas.io/activeredis-instance: s6-dest
name: conn-s6-dest-to-s6-src
namespace: default
resourceVersion: "18872971"
uid: 283ac8fa-d693-46ff-989f-68d018888584
spec:
addresses:
- 192.168.1.10:30010
instance: s6-dest
pause: false
teardownPolicy: Detach
status:
instance: s6-dest
shards:
- index: 0
  offset: "0"
  opId: "0"
  status: Connected
  syncStatus: PartialSync
status: Healthy
upstreamPeer:
  service_id: 0
  service_metadata:
    instance: default/s6-src

Here status.shards[0].status indicates the connection status of the shard, and status.shards[0].syncStatus indicates the data synchronization status. The synchronization status can be PartialSync (indicating incremental synchronization of Oplog) or FullSync (indicating that RDB is being synchronized).

The native Redis Sentinel mode does not have the concept of shards. Here, a master-slave of Sentinel is abstracted as shard 0 to be compatible with the same data structure as the Redis cluster mode.

Pre-flight inspection

Before a connection is accepted, the platform runs a set of pre-flight checks against the upstream. The Web Console exposes them through the Inspect button; they also run automatically during ActiveRedisConnection admission, and a failing check rejects the connection with the corresponding message. The checks are backed by the ActiveRedisInspection resource.

Check ItemCheck ContentHandling Method
Network connectionNetwork connectivity check, password checkConfirm that the upstream address is correct; the address is accessible from the downstream side; the credential is correct.
ArchitecturesRedis instance architecture checkConfirm that the instance architectures of the upstream and downstream instances are consistent.
Cluster mode slices inspectCluster mode shard number and slot distribution checkConfirm that the number of shards and slot distribution of the upstream and downstream instances are the same. If not, you can refer to Initialize Cluster Instance Slot Distribution to create a new instance.
Requirements inspectInstance resource rule checkNeed to ensure that the memory resources of the downstream instance must be greater than or equal to the upstream instance.
Config inspectInstance Service ID checkNeed to ensure that in the replication group, the ServiceID is unique and within the range [0-15].

On Redis 7.2 the inspection dials with the peer-auth credential the data path will actually use, so a successful inspection also confirms that both datacenters carry the same credential. It additionally performs a plain TCP dial against the endpoint it resolves for the replication stream — the address in spec.addresses[0], unless spec.peerPort splits the two.

Removing a connection

Deleting an ActiveRedisConnection tears the link down according to its spec.teardownPolicy:

PolicyEffectReversible
Detach (default)Aborts and removes the peer link. The module keeps the peer's bookkeeping, so re-creating the connection later resumes from the retained state.Yes
DecommissionPermanently decommissions the peer. In addition to removing the link, the module stops retaining Oplog, dead-key tombstones, and lag accounting for that peer.No
Decommission is one-way

Decommission is the correct teardown for a datacenter that is gone for good — a detached-but-never-returning peer keeps holding garbage-collection floors on the survivors. It cannot be changed back to Detach, and a decommissioned peer that later returns is treated as a brand-new peer requiring a fresh full synchronization.

Credential rotation

On Redis 7.2, peer links carry no inline credential: outbound dials read the node-local peer-auth credential that the operator re-pushes on every reconcile. To rotate, update the password Secret of each datacenter's peer-auth RedisUser in place, to the same new value, one datacenter at a time. The RedisUser controller replays the ACL, the operator re-pushes the credential, and the proxy accepts the new credential on the next rebuild. Existing links stay healthy, and a link that drops after the rotation re-dials with the current credential.

Rotating an instance's own password Secret is independent and does not affect peer links.