Music sequence

dasdasdasdMusic in Super Mario 64 is controlled by sequence files. They are in .m64 format, which is similar to MIDI. A sequence file has a sequence script, that can spawn channels, which have channel scripts, that can spawn layers (also known as tracks), which have layer scripts that can play notes. Each note is performed using an instrument from a sound bank. A sequence file can use multiple banks. The mapping from sequences to sound banks is described in sound/sequences.json. Channels can switch between banks using a command. However, in practice most sequences limit themselves to a single sound bank. The main exception is sequence 0, which is responsible for sound effects.

A sequence may play up to 16 channels with up to 4 layers per channel. Several notes can be played simultaneously on a single layer.

A sequence is played as long as the sequence script is not ended. A channel (including layers) is active as long as the channel script is not ended.

Note properties

A note can be played by the commands layer_note0, layer_note1 and layer_note2 (only valid if chan_largenoteson has been called on the layer's channel) or layer_smallnote0, layer_smallnote1 and layer_smallnote2 (only valid if chan_largenotesoff has been called on the layer's channel; default state). A note is defined by the following properties:

Pitch

Also known as note. The pitch defines the frequency in which the instrument is to be played. See list of notes below for available pitches.

Play percentage

Also known as timestamp. This defines the note value, such as half or quarter note. This also implies a layer delay of the same amount. See note values below for possible values.

Velocity

This is the volume of the note.

Duration

Also known as gate or gate time. A value of 0 menas the note is heard during the whole percentage, 128 is half of the percentage and 255 almost none.

List of notes

This is a list of notes as used by the layer_note# and layer_smallnote# commands.

Octave	Note	Decimal	Hex (note0)	Hex (note1)	Hex (note2)
0	A	0	0x00	0x40	0x80
	A♯/B♭	1	0x01	0x41	0x81
	B	2	0x02	0x42	0x82
1	C	3	0x03	0x43	0x83
	C♯/D♭	4	0x04	0x44	0x84
	D	5	0x05	0x45	0x85
	D♯/E♭	6	0x06	0x46	0x86
	E	7	0x07	0x47	0x87
	F	8	0x08	0x48	0x88
	F♯/G♭	9	0x09	0x49	0x89
	G	10	0x0A	0x4A	0x8A
	G♯/A♭	11	0x0B	0x4B	0x8B
	A	12	0x0C	0x4C	0x8C
	A♯/B♭	13	0x0D	0x4D	0x8D
	B	14	0x0E	0x4E	0x8E
2	C	15	0x0F	0x4F	0x8F
	C♯/D♭	16	0x10	0x50	0x90
	D	17	0x11	0x51	0x91
	D♯/E♭	18	0x12	0x52	0x92
	E	19	0x13	0x53	0x93
	F	20	0x14	0x54	0x94
	F♯/G♭	21	0x15	0x55	0x95
	G	22	0x16	0x56	0x96
	G♯/A♭	23	0x17	0x57	0x97
	A	24	0x18	0x58	0x98
	A♯/B♭	25	0x19	0x59	0x99
	B	26	0x1A	0x5A	0x9A
3	C	27	0x1B	0x5B	0x9B
	C♯/D♭	28	0x1C	0x5C	0x9C
	D	29	0x1D	0x5D	0x9D
	D♯/E♭	30	0x1E	0x5E	0x9E
	E	31	0x1F	0x5F	0x9F
	F	32	0x20	0x60	0xA0
	F♯/G♭	33	0x21	0x61	0xA1
	G	34	0x22	0x62	0xA2
	G♯/A♭	35	0x23	0x63	0xA3
	A	36	0x24	0x64	0xA4
	A♯/B♭	37	0x25	0x65	0xA5
	B	38	0x26	0x66	0xA6
4	C	39	0x27	0x67	0xA7
	C♯/D♭	40	0x28	0x68	0xA8
	D	41	0x29	0x69	0xA9
	D♯/E♭	42	0x2A	0x6A	0xAA
	E	43	0x2B	0x6B	0xAB
	F	44	0x2C	0x6C	0xAC
	F♯/G♭	45	0x2D	0x6D	0xAD
	G	46	0x2E	0x6E	0xAE
	G♯/A♭	47	0x2F	0x6F	0xAF
	A	48	0x30	0x70	0xB0
	A♯/B♭	49	0x31	0x71	0xB1
	B	50	0x32	0x72	0xB2
5	C	51	0x33	0x73	0xB3
	C♯/D♭	52	0x34	0x74	0xB4
	D	53	0x35	0x75	0xB5
	D♯/E♭	54	0x36	0x76	0xB6
	E	55	0x37	0x77	0xB7
	F	56	0x38	0x78	0xB8
	F♯/G♭	57	0x39	0x79	0xB9
	G	58	0x3A	0x7A	0xBA
	G♯/A♭	59	0x3B	0x7B	0xBB
	A	60	0x3C	0x7C	0xBC
	A♯/B♭	61	0x3D	0x7D	0xBD
	B	62	0x3E	0x7E	0xBE
6	C	63	0x3F	0x7F	0xBF

Note durations

The effective duration of a note and the delay commands is based on the currently set sequence tempo. The duration of a single tick in seconds can be calculated using the following formular: time = 1.25 / tempo.

Note	Ticks	Duration (120 BPM)
whole	192	2s
half dotted (2nd + 4th)	144	1.5s
half	96	1s
quarter dottet (4th + 8th)	72	0.75s
quarter (4th)	48	0.5s
eighth dottet (8th + 16th)	36	0.375s
quarter tuplet	32	0.333s
eighth (8th)	24	0.25s
sixteenth dottet (16th + 32th)	18	0.1875s
eigthth tuplet	16	0.1666s
sixteenth (16th)	12	0.125s
32th dottet (32th + 64th)	9	0.09375s
sixteenth tuplet (16th)	8	0.08333s
32th	6	0.0625s
32th tuplet	4	0.041666s
64th	3	0.03125s
64th tuplet	2	0.0208333s

Create sequences

There are several ways for creating new sequence files.

Seq64

Seq64 is a GUI program for editing sequence files for Super Mario 64 and other Nintendo games. It also features full MIDI importing and exporting.

https://github.com/sauraen/seq64

Manually

Sequence files can also be created manually in a textual format. Create a new assembly file in the sequences directory, for example sound/sequences/us/23_custom_sequence.s. This file should begin with the following header:

.include "seq_macros.inc"
.section .rodata
.align 0

Next you can write down a series of commands. See include/seq_macros.inc or https://hackmd.io/opEB-OmxRa26P8h8pA-x7w for a list of available commands.

Commands

Sequence commands

Mnemonic	Param 1	Param 2	Bytecode (US)	Description
seq_end			0xff	End of script, loop or function
seq_delay1			0xfe	Delay for 1 tick
seq_delay	ticks		0xfd, var	Delay for X ticks
seq_call	address		0xfc, addr	Call a function
seq_jump	address		0xfb, addr	Jump to a point in the sequence script
seq_beqz	address		0xfa, addr	Jump if Q == 0, where Q is an s8 used to hold temporary sequence script state
seq_bltz	address		0xf9, addr	Jump if Q < 0
seq_loop	iterations		0xf8, u8	Repeat until `loopend` for X iterations (256 if X = 0)
seq_loopend			0xf7	Loop end marker
seq_bgez	address		0xf5, addr	Jump if Q >= 0
seq_reservenotes	numNotes		0xf2, u8	Reserve X notes for exclusive use by this sequence (dropping earlier reservations). A limited number of notes can be played at once (configurable per level, usually 16); if more than that are played, an existing one will be stopped and reused. This command can prevent that from happening across sequences.
seq_unreservenotes			0xf1	Drop all earlier reservations made by this sequence.
seq_transpose	semitones		0xdf, s8	Set transposition in semitones.
seq_transposerel	semitones		0xde, s8	Change transposition by some delta.
seq_settempo	tempo		0xdd, u8	Set tempo in BPM.
seq_addtempo	tempo		0xdc, s8	Change tempo by some delta.
seq_setvol	volume		0xdb, u8	Set volume scale (0…255 = none to twice normal).
seq_changevol	volume		0xda, s8	Change volume scale by some delta.
seq_initchannels	channelMask		0xd7, u16	Initialize channels (bitmask). This copies mute behavior and note allocation policy.
seq_disablechannels	channelMask		0xd6, u16	Uninitialize channels (bitmask).
seq_setmutescale	mulitplier		0xd5, s8	Set volume multiplier for muted mode (0…127).
seq_mute			0xd4	Set sequence to muted mode. Depending on what behavior has been set this will do different things; from lowering volume to not running scripts.
seq_setmutebhv	bitmask		0xd3, u8	Set mute behavior (bitmask). Bit 0x20 lowers volume, bit 0x40 does something to layers, and bit 0x80 pauses the sequence script. Default mute behavior is to have all of those bits set.
seq_setshortnotevelocitytable	address		0xd2, addr	Set velocity table for short notes (see further down). Target should be an array of bytes in the sequence data of length 16.
seq_setshortnotedurationtable	address		0xd1, addr	Set duration table for short notes (see further down). Target should be an array of bytes in the sequence data of length 16.
seq_setnoteallocationpolicy	policyMask		0xd0, u8	Set note allocation policy (bitmask). If bit 0x1 is set, it will try to steal notes back from what was previously played on the same layer. If bit 0x2 is set, notes will be allocated exclusively from the channel’s reserved notes, else if 0x4 is set, from the channel’s or sequence’s reserved notes, else if 0x8 is set, from unreserved notes, otherwise from all sources. Not used by the game.
seq_setval	value		0xcc, u8	Set Q to a constant.
seq_bitand	value		0xc9, u8	Bitand Q by a constant.
seq_subtract	value		0xc8, u8	Subtract a constant from Q.
seq_startchannel	channel	address	0x90-0x9f, addr	Start channel N, with a given channel script.
seq_getvariation			0x80-0x8f	Set Q to the variation bit, which is initially either 0 or 0x80 (i.e. -128, since Q is s8). This bit comes from the sequence load command. For instance, to play the Koopa shell music, the game plays sequence 0x8E, which is the same as sequence 0xE but with the variation bit set.
seq_setvariation			0x70-0x7f	Set the variation bit to Q.
seq_subvariation			0x50-0x5f	Subtract the variation bit from Q.
seq_testchdisabled	channel		0x00-0x0f	Set Q to 1 or 0, depending on whether channel N has been disabled by channel script.

Channel commands

Mnemonic	Param 1	Param 2	Bytecode (US)	Description
chan_end			0xff	End of script, loop or function
chan_delay1			0xfe	Delay for 1 tick
chan_delay	ticks		0xfd, var	Delay for X ticks
chan_call	address		0xfc, addr	Call a function
chan_jump	address		0xfb, addr	Jump to a point in the channel script
chan_beqz	address		0xfa, addr	Jump if Q == 0, where Q is an s8 used to hold temporary channel script state
chan_bltz	address		0xf9, addr	Jump if Q < 0
chan_loop	iterations		0xf8, u8	Repeat until `loopend` for X iterations (256 if X = 0)
chan_loopend			0xf7	Loop end marker
chan_break			0xf6	Decrease loop/function stack depth by 1. Combine with a jump to break out of a loop.
chan_bgez	address		0xf5, addr	Jump if Q >= 0
chan_hang			0xf3	Infinite delay.
chan_reservenotes	numNotes		0xf2, u8	Reserve X notes for exclusive use by this channel (dropping earlier reservations).
chan_unreservenotes	numNotes		0xf1	Drop all earlier reservations made by this channel.
chan_dyncall			0xe4	If Q != -1, make a function call to position dyntable[Q], where dyntable is a channel-specific u16 array which can set by `setdyntable`/`dynsetdyntable`.
chan_setvibratodelay			0xe3, u8	Set delay until vibrato starts for each note.
chan_setvibratoextentlinear			0xe2, u8, u8, u8	Set vibrato extent for each note as a function that starts at X, goes up linearly to Y over a time span given by Z, then stays there.
chan_setvibratoratelinear			0xe1, u8, u8, u8	Like above except for vibrato rate.
chan_setvolscale	volume		0xe0, u8	Set scale factor for volume (0…255 = none to twice normal).
chan_setvol	volume		0xdf, u8	Set scale factor #2 for volume (0…255 = none to twice normal). The two scale factors are separately controlled by outer audio logic, e.g. when changing between rooms, when playing sound effects, etc. The game almost exclusively uses setvol, so that should probably be preferred, but setvolscale shows up in a few locations in the sound effect sequence.
chan_freqscale	value		0xde, u16	Pitch bend using a raw frequency multiplier X/2^15.
chan_setpan	pan		0xdd, u8	Set the pan for this channel (0…128)
chan_setpanchanweight			0xdc, u8	The pan for each note will be computed as W * channel pan + (1 - W) * note pan. This sets W (0…128 maps to 0…1). W defaults to 1.0.
chan_transpose	semitones		0xdb, s8	Set transposition in semitones.
chan_setenvelope	address		0xda, addr	Set ADSR envelope. See below for format.
chan_setdecayrelease	rate		0xd9, u8	Set ADSR decay/release rate.
chan_setvibratoextent	extent		0xd8, u8	Set vibrato extent.
chan_setvibratorate	rate		0xd7, u8	Set vibrato rate.
chan_setupdatesperframe			0xd6, u8	Seems like it was meant to set the number of updates per frame? But it doesn’t actually do anything.
chan_setreverb	reverb		0xd4, u8	Set amount of reverb (or “dry/wet mix”?).
chan_pitchbend	bend		0xd3, s8	Pitch bend by <= 1 octave in either direction (-127…127).
chan_setsustain	volume		0xd2, u8	Set ADSR sustain volume.
chan_setnoteallocationpolicy	policyMask		0xd1, u8	Set note allocation policy for channel. See the description for sequence scripts.
chan_stereoheadseteffects	enable		0xd0, u8	Enable (1) or disable (0) some pan/volume effects related to stereo/headset. I don’t know which ones precisely – pan is still taken into account even with this disabled. With this disabled, stereo/headset gets the same output.
chan_setval	value		0xcc, u8	Set Q to a constant.
chan_readseq	address		0xcb, addr	Set Q to sequenceData[X + Q].
chan_setmutebhv			0xca, u8	Set mute behavior for this channel.
chan_bitand	value		0xc9, u8	Bitand Q by a constant.
chan_subtract	value		0xc8, u8	Subtract a constant from Q.
chan_writeseq		address	0xc7, u8, addr	Overwrite the byte at the given position in the sequence script by (Q + X).
chan_setbank	bank		0xc6, u8	Switch bank within instrument bank sets, to the X’th bank, 0-indexed, counting from the end.
chan_dynsetdyntable			0xc5	If Q != -1, set dyntable to point to position dyntable[Q] in the sequence data.
chan_largenoteson			0xc4	Set notes for this channel to use “large notes” format. SM64 does this for all channels. (See `note` vs. `shortnote` in the layer section below.)
chan_largenotesoff			0xc3	Set notes for this channel to use “short notes” format.
chan_setdyntable	address		0xc2, addr	Set dyntable to point to a given u16 array from the sequence data.
chan_setinstr	instrument		0xc1, u8	Set instrument (program?) within bank. 0x80-0x83 set raw waves (sawtooth, triangle, sine, square), 0x7f is special, otherwise 0-indexed.
chan_dynsetlayer			0xb0-0xbf	If Q != -1, start layer N, with layer script given by dyntable[Q].
chan_freelayer	layer		0xa0-0xaf	Stop layer N.
chan_setlayer	layer	address	0x90-0x9f, addr	Start layer N, with layer script starting at address X.
chan_ioreadval			0x80-0x8f	Set Q to IO[N], where IO is an array of bytes that can be read/written by both the game and the channel script. If N < 4, IO[N] is then set to -1.
chan_iowriteval			0x70-0x7f	Set IO[N] to Q.
chan_setnotepriority			0x60-0x6f	Set note priority for later notes played on this channel to N. When the game wants to play a note and there aren’t any free notes available, a note with smaller or equal priority is discarded. Should be >= 2; 0 and 1 have internal meaning. Defaults to 3.
chan_ioreadvalsub			0x50-0x5f	Decrease Q by IO[N].
chan_ioreadval2			0x40-0x4f, u8	Set Q to IO2[X], where IO2 is the IO array for channel N.
chan_iowriteval2			0x30-0x3f, u8	Set IO2[X] to Q, where IO2 is the IO array for channel N.
chan_disablechannel	channel		0x20-0x2f	Disable channel N for the parent sequence.
chan_startchannel	channel	address	0x10-0x1f, addr	Start channel N for the parent sequence, with channel script starting at address X.
chan_testlayerfinished	layer		0x00-0x0f	Set Q to 1 or 0, depending on whether layer N has been disabled (either forcibly or by finishing its script).

Layer commands

Mnemonic	Param 1	Param 2	Param 3	Param 4	Bytecode (US)	Description
layer_end					0xff	End of script, loop or function
layer_call	address				0xfc, addr	Call a function
layer_jump	address				0xfb, addr	Jump to a point in the layer script
layer_loop	iterations				0xf8, u8	Repeat until `loopend` for X iterations (256 if X = 0)
layer_loopend					0xf7	Loop end marker
layer_setshortnotedurationfromtable					0xe0-0xef	Set the duration for future short notes to table[N], where table defaults to `{229, 203, 177, 151, 139, 126, 113, 100, 87, 74, 61, 48, 36, 23, 10, 0}` and can be set by `setshortnotedurationtable`.
layer_setshortnotevelocityfromtable					0xd0-0xdf	Set the velocity for future short notes to table[N], where table defaults to `{12, 25, 38, 51, 57, 64, 71, 76, 83, 89, 96, 102, 109, 115, 121, 127}` and can be set by `setshortnotevelocitytable`.
layer_setpan	pan				0xca, u8	Set the pan for this layer (0…128).
layer_setshortnoteduration	duration				0xc9, u8	Set the duration for future short notes.
layer_disableportamento					0xc8	Disable portamento for later notes.
layer_portamento					0xc7, u8, u8, u8/var	Enable portamento (aka glissando; continuously sliding pitch). X describes the mode of operation somehow using its 0x80 bit and a value 0-5 in its lower-order bits. Y describes the target pitch. Z gives the duration, and is a u8 if the 0x80 bit is set, otherwise a var.
layer_setinstr	instrument				0xc6, u8	Set instrument/program. Similar to the channel command, except arguments 0x7f and above are not supported.
layer_somethingoff					0xc5	Turn off some bool related to how notes decay somehow? (This is the default.)
layer_somethingon					0xc4	Turn on the same thing.
layer_setshortnotedefaultplaypercentage	percentage				0xc3, var	Set the default play percentage for short notes (see below).
layer_transpose	semitones				0xc2, u8	Set transposition in semitones.
layer_setshortnotevelocity	velocity				0xc1, u8	Set velocity for future short notes.
layer_delay	ticks				0xc0, var	Delay for N ticks
layer_note0	pitch	percentage	velocity	duration	0x00-0x3f, var, u8, u8	Play note with specified pitch, percentage, velocity and duration. Only valid if channel is set to “large notes”.
layer_note1	pitch	percentage	velocity		0x40-0x7f, var, u8	Play note with specified pitch, percentage and velocity and duration of 0. Only valid if channel is set to “large notes”.
layer_note2	pitch	velocity	duration		0x80-0xbf, u8, u8	Play note with specified pitch, velocity and duration. Only valid if channel is set to “large notes”.
layer_smallnote0	pitch	percentage			0x00-0x3f, var	Play note with specified pitch and percentage and velocity and duration set with layer_setshortnotevelocity and layer_setshortnoteduration. Only valid if channel is set to “small notes”.
layer_smallnote1	pitch				0x40-0x7f	Play note with specified pitch and the default play percentage, as set by layer_setshortnotedefaultplaypercentage (there is no built-in default; using it without that command will read uninitialized memory). Velocity and duration is set like in layer_smallnote0. Only valid if channel is set to “small notes”.
layer_smallnote2	pitch				0x80-0xbf	Play note with specified pitch and the previous play percentage. Velocity and duration is set like in layer_smallnote0. Only valid if channel is set to “small notes”.