Some learnings in creating a music game for game jam. And appreciated all the feedback.

Key topics:

• Handling musical notes and pitch
• Multiple cinemachine cameras and using a "ghost" middle player to track
• C# Enumerated types
• Scrolling UI text field
• Using raycasts and containers to guide AI to specific targets within a set of objects
• Jam lessons learned

https://baylight.itch.io/road2joy

Game summary:

• Navigate through musical notes for the tune "Ode to Joy"
• 1-2 players. If 1 player, the "AI" chooses the harmony notes.
• There are "correct" notes and "incorrect" notes
• Hitting an incorrect note causes player to rewind to the last checkpoint
• Checkpoints are the musical measures
• Notes turn color based on the player that touched them and plays sound. Notes turn red if wrong.
• Hard mode has high speed and no showing successful notes if player has to replay measure.
• Play through the end and then hear the full song plus players can swap paths
• Hearts appear when players cross paths

Technical approaches and highlights:

• Notes are gameobjects which have a correct/incorrect status, a note value (C4, G3) corresponding to their pitch. and a display string such as "C" or "G" for the note name.
• When player collides with note, if its correct gameplay continues, if not it triggers an event for a wrong note
• A Gamemanager handles most events, although towards the end of the jam I was shortcutting object references.
• I learned that in many music games and synthesizers, you don't load separate note sounds (just like not loading individual sprite graphics but use sprite sheet). The convention is to have a few notes or note and then vary the pitch. This worked very well. I loaded a "middle C" note and then adjusted the pitch. It took a while to get right. There is in theory a fixed relative value for note steps, such as C to D, etc. I didn't get perfect but sort of found some that worked and used as baseline for others.
• I have a struct enum that has the various note to pitch comparisons. So C4 had pitch of 1.0 (since it was the note I loaded). This had one issue listed below in learnings. For example, used middle C (C4) and then changed pitch for other notes.
• C4: 1.0f
  
• D4: 1.12f
• E4: 1.25f
• F4: 1.33f

• Checkpoints are based on measures. Used a hidden (0 alpha) checkpoint and the player has a "last_measure" which resets once reached. Each measure has a visual, a checkmark (which somehow got messed up with order layers), and spawn points for the players.
• I struggled but am proud of the approach to the harmony "AI". So the way the computer harmony works is:
• Musical Note gameobjects are children of a note "Set" gameobject that serves as a collection.
• To figure out how to harmonize, the AI player sends out a raycast to find the next Set. When found, it asks the set for the highest and lowest correct note. The AI player checks are they positioned higher or lower than Player 1. If higher, they take the path towards the higher correct note. If lower, they choose the lower correct note. This creates the "harmony" by balancing the actions of player 1. The Set also has a collider child at the edge so once the AI player goes past it, it then raycasts again to find the next set. This probably could have been done by also checking the rect size of the Set. But overall this worked well to group objects together and then have the AI raycast and the set reports the correct notes. 
• Using cinemachine between two players: it worked out well to create a third "ghost" player, that has alpha=0 and is invisible. Its position is always in between the two players and then I have the cinemachine camera track the "ghost". This worked really well to scale the view and include the two players and position nicely in between.

Technical learnings:

• I didn't realize C#/Unity enums in the inspector keep the original value. So I had a enum list of C4, D4, E4. In the inspector, I assigned the note gameobject the enum value C4. But then I wanted to add B3 ahead in the list (B3, C4, D4...) This messed things up. I thought the inspector would store "C4" and then map it to the second item. But it seems if I understood that once I assigned C4, it stored it as enum value 0. So adding B3 meant that the note with "C4" had stored 0 so it picked up B3. Basically if I understand correctly, enums use integer values and inserting a new enum can mess things up because the Unity inspector is storing the sequence number and not the name. 
• Originally I wanted to autogenerate the notes from a file. I was going to use some basic notation of "C4" to then convert and auto-space out the notes. But ran out of time so placed the notes visually in the scene and then marked them as "IsCorrectNote" boolean.
• Humbled how long debugging took. I had many plans to introduce fun obstacles like a "PacMan" that eats notes, other notes that fall and player have to pause to let pass before continuing, and even a "Beethoven Boss" in Hardmode that would shoot out sharps.
• Learned about "ignore raycast" layers which was super helpful so AI wouldn't hit other items it wasn't interested in.
• Not 100% sure I did right, but I learned about changing Navigation UI settings to "None" so spacebar wouldn't select focused button.
• Still learning about UI canvas to do layout right, but some raw notes: "UI - click on Rect and pin to place like bottom left. On Render mode choose “Screen Space - Overlay” and on Canvas Scaler choose “Scale with Screen Size” to handle regular and HD"
• Had a ridiculously hard time making a scrolling text viewer. Despite using many tutorials, some class between code, hiearchy, and settings in inspector never worked. This tutorial finally worked for me:
• Had trouble with layer ordering. For some reason things would work and then when doing the WebGL and Windows builds, the orders became more "sensitive" so I lost some visual elements that layered differently on my Mac. Need to figure out if ordering was right strategy or should have used offset visual layers in 3d.
• Big learning was I wanted to have 2 camera views - once was main screen and then zoom into notes on the desk. At first I used one camera and tries to change the Ortho Size. but this didn't work even if I tried to dynamically update. Also annoying was I couldn't seem to change the main camera transforms once I attached the cinemachine and had to unattach to reset then reattach. What eventually worked was two cinemachine cameras that by default are disabled. Each has its own config and I enable/disable accordingly. This worked way better than trying to change the settings on one camera programmatically.
  
  using UnityEngine;
  using Cinemachine; 
  public class CameraController : MonoBehaviour
  {    
  public CinemachineVirtualCamera deskCamera;    
  public CinemachineVirtualCamera gameCamera;    
  public GameObject startButtonPanel;    
  void Start()    
  {        
  deskCamera.gameObject.SetActive(true);      
   gameCamera.gameObject.SetActive(false);
    }  
    void Update()    
  {         if (Input.GetKeyDown(KeyCode.Return))  
        {             ZoomIn();             startButtonPanel.gameObject.SetActive(false);         }
  ...

Game design learnings (and thank you for people that played and commented so I learned)

• Everything takes longer than expected
• Play test early
• Don't lose track of the beginner player. Should have made it easier to start, like either simpler note level, or highlighting correct notes to get the player used to the song.
• Originally I was going to just have the player travel up or down but then thought having other notes might work to learn. But I think it became a bit too niche for someone who knows something about music already. To introduce more slowly could have taught.
• There were bugs if player skipped notes it didn't stop. It was better to skip than hit wrong note. I ran out of time to address.
• As always, the game mechanic is most important and I would spend more time on this before jumping into other visuals.
• Take more breaks. Tendency to push through working and sometimes break was a good reset to better solve problems.
• It was hard work but fun.