I needed to get Particular Service Control up and running on our k8s cluster this week. Part of that is to get an instance of RavenDB running in the cluster and this actually caused me a bit of trouble. I kept running into problems where RavenDB would start up but then report that it couuld not access the data directory. What was up?
I tried overriding the entry point for the container and attaching to it to see what was going on but I couldn’t see anything wrong. I was able to write to the directory without issue. Eventually I stumbled on a note in the RavenDB documentation which mentioned a change in the 6.x version of RavenDB which meant that Raven no longer ran as root inside the container.
K8S has the ability to change the ownership of the volume to the user that the container is running as. This is done by setting the fsGroup property in the pod spec. In this case Raven runs as UID 999. So I updated my tanka spec to include the fsGroup property and the problem was solved.
...
deployment: deployment.new($._config.containers.ravendb.name) {
metadata+: {
namespace: 'wigglepiggle-' + $._config.environment,
labels: {
app: $._config.containers.ravendb.name,
},
},
spec+: {
replicas: 1,
selector: {
matchLabels: $._config.labels,
},
template: {
metadata+: {
labels: $._config.labels,
},
spec+: {
securityContext: {
fsGroup: 999,
fsGroupChangePolicy: 'OnRootMismatch',
},
containers: [
{
name: $._config.containers.ravendb.name,
image: $._config.containers.ravendb.image,
ports: [{ containerPort: 8080 }, { containerPort: 38888 }],
volumeMounts: [
{
name: 'data',
mountPath: '/var/lib/ravendb/data',
},
],
env: [
{
name: 'RAVEN_Setup_Mode',
value: 'None',
},
{
name: 'RAVEN_License_Eula_Accepted',
value: 'true',
},
{
name: 'RAVEN_ARGS',
value: '--log-to-console',
},
{
name: 'RAVEN_Security_UnsecuredAccessAllowed',
value: 'PrivateNetwork',
},
],
},
],
volumes: [
{
name: 'data',
persistentVolumeClaim: {
claimName: $._config.containers.ravendb.name,
},
},
],
},
},
},
},
...
This generated yml like
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: service-control-ravendb
name: service-control-ravendb
namespace: wigglepiggle-dev
spec:
replicas: 1
selector:
matchLabels:
app: service-control
environment: dev
template:
metadata:
labels:
app: service-control
environment: dev
spec:
containers:
- env:
- name: RAVEN_Setup_Mode
value: None
- name: RAVEN_License_Eula_Accepted
value: "true"
- name: RAVEN_ARGS
value: --log-to-console
- name: RAVEN_Security_UnsecuredAccessAllowed
value: PrivateNetwork
image: ravendb/ravendb:6.0-latest
name: service-control-ravendb
ports:
- containerPort: 8080
- containerPort: 38888
volumeMounts:
- mountPath: /var/lib/ravendb/data
name: data
securityContext:
fsGroup: 999
fsGroupChangePolicy: OnRootMismatch
volumes:
- name: data
persistentVolumeClaim:
claimName: service-control-ravendb
