gcp-hockey-results/motm_app/PRODUCTION_MIGRATION_GUIDE.md

Production Database Migration Guide

Issue

The production database is missing the votingdeadline column, causing SQL errors when accessing the MOTM admin page.

Solution

Run the migration script on the production Kubernetes cluster to add the missing column.

Step-by-Step Instructions

Option 1: Run Migration via kubectl exec (Recommended)

1. Find your production pod:

   kubectl get pods -n <your-namespace> -l app.kubernetes.io/name=motm-app
   

   Replace <your-namespace> with your actual namespace (e.g., motm-app, default, etc.)

2. Execute the migration script:

   kubectl exec -it <pod-name> -n <your-namespace> -- python add_voting_deadline.py
   

   Example:

   kubectl exec -it motm-app-7d8f9b4c5-xk2mn -n motm-app -- python add_voting_deadline.py
   

3. Verify the migration:

   kubectl exec -it <pod-name> -n <your-namespace> -- python -c "
   from db_config import db_config
   from sqlalchemy import text, inspect
   engine = db_config.engine
   inspector = inspect(engine)
   columns = inspector.get_columns('admin_settings')
   voting_deadline_exists = any(col['name'] == 'votingdeadline' for col in columns)
   print('✓ votingdeadline column exists' if voting_deadline_exists else '✗ votingdeadline column missing')
   "
   

Option 2: Run Migration via Helm Job (Alternative)

If you prefer to run the migration as a Kubernetes job:

1. Create a migration job manifest:

   # migration-job.yaml
   apiVersion: batch/v1
   kind: Job
   metadata:
     name: motm-migration-voting-deadline
     namespace: motm-app
   spec:
     template:
       spec:
         containers:
         - name: migration
           image: your-registry/motm-app:latest
           command: ["python", "add_voting_deadline.py"]
           envFrom:
           - secretRef:
               name: motm-app-secrets
           - configMapRef:
               name: motm-app-config
         restartPolicy: Never
     backoffLimit: 3
   

2. Apply the job:

   kubectl apply -f migration-job.yaml
   

3. Check job status:

   kubectl get jobs -n motm-app
   kubectl logs -n motm-app job/motm-migration-voting-deadline
   

4. Clean up after successful migration:

   kubectl delete job motm-migration-voting-deadline -n motm-app
   

Option 3: Run Migration via Helm Hook (Advanced)

For automated migrations during deployments:

1. Create a migration job template:

   # templates/migration-job.yaml
   apiVersion: batch/v1
   kind: Job
   metadata:
     name: {{ include "motm-app.fullname" . }}-migration
     namespace: {{ .Release.Namespace }}
     annotations:
       "helm.sh/hook": pre-upgrade,pre-install
       "helm.sh/hook-weight": "-5"
       "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
   spec:
     template:
       spec:
         containers:
         - name: migration
           image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
           command: ["python", "add_voting_deadline.py"]
           envFrom:
           - secretRef:
               name: {{ include "motm-app.fullname" . }}-secrets
         restartPolicy: Never
     backoffLimit: 3
   

2. The migration will run automatically during helm upgrade

Verification

After running the migration, verify it worked:

1. Check the application logs:

   kubectl logs -n <your-namespace> -l app.kubernetes.io/name=motm-app --tail=50
   

2. Test the admin page:

   • Navigate to https://motm.ervine.cloud/admin/motm
   • The page should load without SQL errors
   • You should see the "Voting Deadline" field in the form

3. Check for the column in the database:

   kubectl exec -it <pod-name> -n <your-namespace> -- python -c "
   from db_config import db_config
   from sqlalchemy import text, inspect
   engine = db_config.engine
   inspector = inspect(engine)
   columns = inspector.get_columns('admin_settings')
   print('Columns in admin_settings:')
   for col in columns:
       print(f'  - {col[\"name\"]}: {col[\"type\"]}')
   "
   

   You should see votingdeadline in the list.

Troubleshooting

Issue: "pod not found"

Solution: Make sure you're using the correct namespace and pod name.

Issue: "permission denied"

Solution: The migration script needs database write permissions. Ensure your database user has ALTER TABLE permissions.

Issue: "column already exists"

Solution: This is fine! The migration script is idempotent and will skip if the column already exists.

Issue: Migration fails

Solution: Check the pod logs for detailed error messages:

kubectl logs -n <your-namespace> <pod-name>

Rollback (If Needed)

If the migration causes issues, you can rollback:

1. Remove the column:

   kubectl exec -it <pod-name> -n <your-namespace> -- python -c "
   from db_config import db_config
   from sqlalchemy import text
   engine = db_config.engine
   with engine.connect() as conn:
       conn.execute(text('ALTER TABLE admin_settings DROP COLUMN IF EXISTS votingdeadline'))
       conn.commit()
   print('Column removed')
   "
   

2. Restart the application:

   kubectl rollout restart deployment/motm-app -n <your-namespace>
   

Quick Reference

# Find pod name
kubectl get pods -n motm-app -l app.kubernetes.io/name=motm-app

# Run migration
kubectl exec -it <POD_NAME> -n motm-app -- python add_voting_deadline.py

# Verify migration
kubectl exec -it <POD_NAME> -n motm-app -- python -c "from db_config import db_config; from sqlalchemy import text, inspect; engine = db_config.engine; inspector = inspect(engine); columns = inspector.get_columns('admin_settings'); print('✓ votingdeadline exists' if any(c['name'] == 'votingdeadline' for c in columns) else '✗ missing')"

# Check logs
kubectl logs -n motm-app -l app.kubernetes.io/name=motm-app --tail=100

Next Steps

After the migration is complete:

1. ✅ Restart the application pods to clear any cached connections
2. ✅ Test the MOTM admin page
3. ✅ Set a voting deadline for the next match
4. ✅ Test the voting page to see the countdown timer

Support

If you encounter any issues:

1. Check the pod logs for detailed error messages
2. Verify database connectivity
3. Ensure the database user has proper permissions
4. Review the application logs for any additional errors