MLflow Experiment Tracking Server

MLflow 3.17.2 records experiment parameters, metrics and artifacts on this instance. The server runs as mlflow on 127.0.0.1:5000, using separate tracking and authentication SQLite databases and local artifact storage. CPU only; no model provider is needed. Start with a t3.medium, 4 GiB RAM and 32 GiB storage. Software is USD 0.03 per running instance-hour; EC2, EBS and transfer cost extra.

After launch, allow about two minutes for the generated accounts and service to initialize. Connect using Session Manager or SSH as ubuntu, then run:

sudo -iu ubuntu bash /opt/sushi/docs/demo.sh

The demo creates one synthetic experiment, logs alpha=0.25 and score=0.125, uploads a text artifact and independently retrieves its bytes and SHA-256. Results are grouped under ~/mlflow-results/. Keep the printed manifest path:

sushi-mlflow read ~/mlflow-results/RESULT-DIRECTORY/manifest.json

That read command creates no new run. Use it after a restart or reboot to check the same stored result.

For the browser UI, forward your workstation's port 5000 to this instance's 127.0.0.1:5000 using an authenticated SSH or Session Manager tunnel. Open http://localhost:5000. No inbound database rule is required. This HTTP endpoint must remain behind the tunnel; do not expose it directly to a network.

First boot generates a customer account in ~/.config/sushi/mlflow/access.json, readable only by ubuntu. The helper reads it automatically. Open that file in your SSH session's editor to use the browser sign-in; never copy its contents to shared logs. The separate administrator identity is /etc/mlflow/owner.json, accessible only through root/sudo. Accounts, the CSRF key and databases are generated after launch and preserved on reboot. The service receives the administrator password only during account bootstrap.

Uploaded artifacts are stored data. This setup does not serve uploaded models or execute pickles. Server job execution, periodic job tasks and custom scorers are disabled. One instance is intended for a small tracking workload; SQLite is not a distributed or high-availability database.

Operational commands:

sudo systemctl status mlflow sushi-mlflow-firstboot sushi-mlflow-ready
sudo journalctl -u mlflow --since '10 minutes ago'
sudo systemctl restart mlflow

If initial setup fails, retain /etc/mlflow, /var/lib/mlflow and the owner-only /var/log/mlflow/bootstrap-*.log files. Do not delete databases or regenerate an identity as a repair. Correct the reported error, then restart the firstboot and application units. A forgotten administrator password needs an authenticated rotation/recovery procedure, rather than a new database.

Back up SQLite, artifacts and identities together while mlflow.service is stopped. Include /var/lib/mlflow, /etc/mlflow and the customer access file; write the archive with mode 0600 to protected storage. Restore the complete set onto the same application version with the service stopped and preserve numeric ownership and modes. Do not restore only one database. Test readback before resuming writes. Source inputs and delivered notices are under /opt/mlflow and /usr/share/doc/mlflow; installed inventory is under /opt/sushi/evidence.