Supervisor Setup

RhodeCode Control uses Supervisor to monitor and control RhodeCode Enterprise instances.

During installation, RhodeCode Control installs and sets a Supervisor configuration for each instance. The default location is ~/.rccontrol/supervisor/supervisord.ini

Individual instances can be managed from the CLI through the rccontrol start, rccontrol start and rccontrol restart commands see the RhodeCode Control CLI Guide for examples.

Linux Startup Script

To create a startup script to bring RhodeCode Control and all installed RhodeCode Enterprise & VCS Server instances up after a system restart follow these steps.

  1. Ensure that all relevant instances in ~/.rccontrol.ini have start_at_boot = True.
  2. If an instance has self-managed-supervisor = True make sure the entry in ~/.rccontrol/supervisor/supervisord.ini has set autostart = true.
  3. Add the below to a startup script, cron job, or which ever method you prefer to restart programmes after a reboot.
su - user -c "/home/user/.rccontrol-profile/bin/rccontrol self-init"

Note

Ensure that the same user profile is used when calling the startup scripts as was used to install RhodeCode Control otherwise permission issues will arise.

Auto-start using Systemd

If your RhodeCode Enterprise instances are configured to be started automatically once RhodeCode Control is started, as described in the Manually Setting –start-at-boot section, you can then use the following example to configure a systemd service to bring them up after a server reboot.

  1. Create a systemd service file based on the following example, where $user is the user profile used when calling the startup scripts. This should be the same user that was used to install RhodeCode Control.
[Unit]
Description=Rhodecode
[Service]
Type=forking
User=$user
ExecStart=/path/to/.rccontrol-profile/bin/rccontrol-self-init
[Install]
WantedBy=multi-user.target
  1. Enable and start the service.
$ systemctl enable rhodecode
$ systemctl start rhodecode

For more information about configuring systemd, see the Systemd Service Unit Configuration documentation.

Manually Setting --start-at-boot

By default all instances will be installed with this enabled unless you passed --start-at-boot no. This option controls how Supervisor will manage your instance when it starts the related process.

To change how your RhodeCode Enterprise or VCS Server instance is managed follow these steps.

  1. Open the ~/.rccontrol.ini file.
  2. Each instance will have a configuration section inside if it was initially installed with start-at-boot, see the below example.
[instance:enterprise-1]
start_at_boot = True
self_managed_supervisor = False
[instance:vcsserver-1]
start_at_boot = True
self_managed_supervisor = False
  1. Either toggle the value from True to False or create a record for your instance if it is not there.
  2. Once you save the file, run the rccontrol self-init command and you should see Updated process group <instance-id>.

If you want to verify that the changes worked, check that the autostart = true option is set in the instance section of the ~/.rccontrol/supervisor/supervisord.ini file.

[groupenterprise-1]
programs = enterprise-1_script
[program:enterprise-1_script]
numprocs = 1
redirect_stderr = true
environment = PYTHONPATH="",SSL_CERT_FILE="/home/ubuntu/.rccontrol-profile/etc/ca-bundle.crt",REQUESTS_CA_BUNDLE="/home/ubuntu/.rccontrol-profile/etc/ca-bundle.crt",GIT_SSL_CAINFO="/home/ubuntu/.rccontrol-profile/etc/ca-bundle.crt"
directory = /home/user/.rccontrol/enterprise-1
command = /home/user/.rccontrol/enterprise-1/profile/bin/gunicorn --paster=/home/user/.rccontrol/enterprise-1/rhodecode.ini
autostart = true
numprocs_start = 5000

Manually Setting --self-managed-supervisor

By default all instances will be installed with this disabled unless you passed --self-managed-supervisor. This option allows you to customize Supervisor settings for an instance otherwise RhodeCode Control will overwrite your changes.

To change how your RhodeCode Enterprise or VCS Server instance is managed follow these steps.

  1. Open the ~/.rccontrol.ini file.
  2. Each instance will have a configuration section inside if it was initially installed with self-managed-supervisor, see the below example.
[instance:enterprise-1]
start_at_boot = True
self_managed_supervisor = False
[instance:vcsserver-1]
start_at_boot = True
self_managed_supervisor = False
  1. Either toggle the value from False to True or create a record for your instance if it is not there.
  2. Once you save the file, run the rccontrol self-init command.

Warning

Toggling the option to False will result in RhodeCode Control overwriting any custom Supervisor changes when it is initialized!

Using Include statement to extend the programs

It’s sometimes useful to put additional programs that start together with RhodeCode Enterprise. To add such option you need to use supervisord [include] group to add extra configs. From version 1.6.4 of RhodeCode Control it will be added automatically. If you want to add it manually please simply put this inside the ~/.rccontrol/supervisor/supervisord.ini file

[include]
files = rhodecode_config*.ini

This will import any file that starts with rhodecode_config, and has .ini extension. For example to allow Channelstream to be started together with RhodeCode Enterprise simply create a file called ~/.rccontrol/supervisor/rhodecode_config_channelstream.ini with such content:

[program:rhodecode_channelstream]
numprocs = 1
priority = 30
directory = /home/user/.rccontrol/enterprise-1
command = /home/user/.rccontrol/enterprise-1/profile/bin/channelstream --secret=SECRET --port=9800 --host=127.0.0.1
redirect_stderr = true
stdout_logfile = /home/user/.rccontrol/enterprise-1/rhodecode_channelstream.log

or to start Celery, create another file ~/.rccontrol/supervisor/rhodecode_config_celery.ini and create such entry:

[program:rhodecode_celery]
numprocs = 1
priority = 40
directory = /home/user/.rccontrol/enterprise-1
command = /home/user/.rccontrol/enterprise-1/profile/bin/paster celeryd /home/user/.rccontrol/enterprise-1/rhodecode.ini -l debug
redirect_stderr = true
stdout_logfile = /home/user/.rccontrol/enterprise-1/rhodecode_celery.log

Resetting to Default Settings

Hopefully you will never get to this point but if you have messed up the Supervisor settings beyond repair you can manually force reset them by following these steps.

  1. Backup the ~/.rccontrol/supervisor/supervisord.ini file to another location just in case you need to refer to it again.
  2. cat ~/.rccontrol/supervisor/supervisord.pid to show the pid.
  3. kill -9 <pid> to terminate the supervisord process.
  4. rm ~/.rccontrol/supervisor/supervisord.* to remove existing settings.
  5. rccontrol self-init to restart supervisor with default settings.
$ rccontrol self-init
Initializing supervisor.
Starting supervisord.
Supervisord state is: RUNNING
Added process group enterprise-1
Added process group vcsserver-1