.. _supervisor-setup: Supervisor Setup ================ |RCC| uses `Supervisor`_ to monitor and control |RCEE| instances. During installation, |RCC| installs and sets a Supervisor configuration for each instance. The default location is :file:`~/.rccontrol/supervisor/supervisord.ini` * |RCC| will automatically restart Supervisor if a command requires it. * By default all installed/imported |RCEE| instance are set to auto start. To change this see :ref:`set-start-boot`. * By default |RCC| will manage your settings and not allow customizations for the `[instance:*]` groups. To overcome this you can add new instances by using the `[include]` section, see :ref:`using-include-statement`. In order to change the builtin `[instance:*]` config see :ref:`set-self-managed-supervisor`. * If ever you need to see :ref:`supervisor_reset`. * Logs are located at :file:`~/.rccontrol/supervisor/supervisord.log` Individual instances can be managed from the CLI through the ``rccontrol start``, ``rccontrol start`` and ``rccontrol restart`` commands see the :ref:`rcc-cli` for examples. Linux Startup Script -------------------- To create a startup script to bring |RCC| and all installed |RCEE| & |VCS| instances up after a system restart follow these steps. 1. Ensure that all relevant instances in :file:`~/.rccontrol.ini` have ``start_at_boot = True``. 2. If an instance has ``self-managed-supervisor = True`` make sure the entry in :file:`~/.rccontrol/supervisor/supervisord.ini` has set ``autostart = true``. 3. Add the below to a startup script, cron job, or which ever method you prefer to restart programmes after a reboot. .. code-block:: bash su - user -c "/home/user/.rccontrol-profile/bin/rccontrol self-init" .. note:: Ensure that the same user profile is used when calling the startup scripts as was used to install |RCC| otherwise permission issues will arise. .. _auto-sysd: Auto-start using Systemd ------------------------ If your |RCEE| instances are configured to be started automatically once |RCC| is started, as described in the :ref:`set-start-boot` section, you can then use the following example to configure a systemd service to bring them up after a server reboot. 1. Create a systemd service file based on the following example, where ``$user`` is the user profile used when calling the startup scripts. This should be the same user that was used to install |RCC|. .. code-block:: ini [Unit] Description=Rhodecode [Service] Type=forking User=$user ExecStart=/path/to/.rccontrol-profile/bin/rccontrol-self-init [Install] WantedBy=multi-user.target 2. Enable and start the service. .. code-block:: bash $ systemctl enable rhodecode $ systemctl start rhodecode For more information about configuring systemd, see the `Systemd Service Unit Configuration`_ documentation. .. _set-start-boot: Manually Setting ``--start-at-boot`` ------------------------------------ By default all instances will be installed with this enabled unless you passed ``--start-at-boot no``. This option controls how Supervisor will manage your instance when it starts the related process. To change how your |RCEE| or |VCS| instance is managed follow these steps. 1. Open the :file:`~/.rccontrol.ini` file. 2. Each instance will have a configuration section inside if it was initially installed with ``start-at-boot``, see the below example. .. code-block:: ini [instance:enterprise-1] start_at_boot = True self_managed_supervisor = False [instance:vcsserver-1] start_at_boot = True self_managed_supervisor = False 3. Either toggle the value from True to False or create a record for your instance if it is not there. 4. Once you save the file, run the ``rccontrol self-init`` command and you should see ``Updated process group ``. If you want to verify that the changes worked, check that the ``autostart = true`` option is set in the instance section of the :file:`~/.rccontrol/supervisor/supervisord.ini` file. .. code-block:: ini [groupenterprise-1] programs = enterprise-1_script [program:enterprise-1_script] numprocs = 1 redirect_stderr = true environment = PYTHONPATH="",SSL_CERT_FILE="/home/ubuntu/.rccontrol-profile/etc/ca-bundle.crt",REQUESTS_CA_BUNDLE="/home/ubuntu/.rccontrol-profile/etc/ca-bundle.crt",GIT_SSL_CAINFO="/home/ubuntu/.rccontrol-profile/etc/ca-bundle.crt" directory = /home/user/.rccontrol/enterprise-1 command = /home/user/.rccontrol/enterprise-1/profile/bin/gunicorn --paster=/home/user/.rccontrol/enterprise-1/rhodecode.ini autostart = true numprocs_start = 5000 .. _set-self-managed-supervisor: Manually Setting ``--self-managed-supervisor`` ---------------------------------------------- By default all instances will be installed with this disabled unless you passed ``--self-managed-supervisor``. This option allows you to customize Supervisor settings for an instance otherwise |RCC| will overwrite your changes. To change how your |RCEE| or |VCS| instance is managed follow these steps. 1. Open the :file:`~/.rccontrol.ini` file. 2. Each instance will have a configuration section inside if it was initially installed with ``self-managed-supervisor``, see the below example. .. code-block:: ini [instance:enterprise-1] start_at_boot = True self_managed_supervisor = False [instance:vcsserver-1] start_at_boot = True self_managed_supervisor = False 3. Either toggle the value from False to True or create a record for your instance if it is not there. 4. Once you save the file, run the ``rccontrol self-init`` command. .. warning:: Toggling the option to False will result in |RCC| overwriting any custom Supervisor changes when it is initialized! .. _using-include-statement: Using Include statement to extend the programs ---------------------------------------------- It's sometimes useful to put additional programs that start together with |RCEE|. To add such option you need to use supervisord [include] group to add extra configs. From version 1.6.4 of |RCC| it will be added automatically. If you want to add it manually please simply put this inside the :file:`~/.rccontrol/supervisor/supervisord.ini` file .. code-block:: ini [include] files = rhodecode_config*.ini This will import any file that starts with rhodecode_config, and has .ini extension. For example to allow `Channelstream` to be started together with |RCEE| simply create a file called :file:`~/.rccontrol/supervisor/rhodecode_config_channelstream.ini` with such content: .. code-block:: ini [program:rhodecode_channelstream] numprocs = 1 priority = 30 directory = /home/user/.rccontrol/enterprise-1 command = /home/user/.rccontrol/enterprise-1/profile/bin/channelstream --secret=SECRET --port=9800 --host=127.0.0.1 redirect_stderr = true stdout_logfile = /home/user/.rccontrol/enterprise-1/rhodecode_channelstream.log or to start `Celery`, create another file :file:`~/.rccontrol/supervisor/rhodecode_config_celery.ini` and create such entry: .. code-block:: ini [program:rhodecode_celery] numprocs = 1 priority = 40 directory = /home/user/.rccontrol/enterprise-1 command = /home/user/.rccontrol/enterprise-1/profile/bin/paster celeryd /home/user/.rccontrol/enterprise-1/rhodecode.ini -l debug redirect_stderr = true stdout_logfile = /home/user/.rccontrol/enterprise-1/rhodecode_celery.log .. _supervisor_reset: Resetting to Default Settings ----------------------------- Hopefully you will never get to this point but if you have messed up the Supervisor settings beyond repair you can manually force reset them by following these steps. 1. Backup the :file:`~/.rccontrol/supervisor/supervisord.ini` file to another location just in case you need to refer to it again. 2. ``cat ~/.rccontrol/supervisor/supervisord.pid`` to show the pid. 3. ``kill -9 `` to terminate the supervisord process. 4. ``rm ~/.rccontrol/supervisor/supervisord.*`` to remove existing settings. 5. ``rccontrol self-init`` to restart supervisor with default settings. .. code-block:: bash $ rccontrol self-init Initializing supervisor. Starting supervisord. Supervisord state is: RUNNING Added process group enterprise-1 Added process group vcsserver-1 .. _Supervisor: http://supervisord.org/index.html .. _Systemd Service Unit Configuration: http://www.freedesktop.org/software/systemd/man/systemd.service.html