July 01, 2022

Compiling TSE3 on Raspberry Pi

Goal and background

This project is a follow up on my previous project to emulate a floppy disk controller for the Yamaha QX1. The new goal is to study the possibility to replace the entire motherboard with a custom board and a Raspberry Pi.

Doing so should provide:

An upgradable system
USB
STM32F103C8T6 Blue pill as 8 channel UART (bit banging) 8 * ATTiny4313
128 concurrent MIDI channels (8*16), instead of only 8
More and non-volatile memory (with a F-RAM from Cypress Semiconductor such as FM24V05-GTR)
Better workflow with less confirmation message
Ability to connect a PC keyboard, USB Key, Monitor, ...

Identified steps

Compile the sequencer engine (TSE3)
Program a 8 channel bit banging UART (MIDI out) and the embedded UART (for MIDI In) on the Blue pill
Program a basic SPI protocol between the RPI and Blue pill
Connect the STM32 to board JK
Program a basic UI to test MIDI In and Out
Decode the protocol between board DM and PNA
Interface board PNA and the RPI
Program a better UI, mimicking the QX1 workflow and using TSE3
Cherry on sundae: add Sync Out/In

Since I have no time to program a full sequencer, I looked for a sequencer engine with a set of API and I found two possible candidates:

ALSA
TSE3

ALSA

Advanced Linux Sound Architecture is the de-facto official and supported sound achitecture on Linux. It includes the management on sound cards, MIDI and provides a sequencer (aseq). More, some ALSA components such as amixer use ncurses. However, it doesn't seem to be as usable as TSE3. To be developped.

TSE3

TSE3 (Trax Sequencer Engine v3), written by Pete Goodliffe, is an open source project which provides exactly what I need. It's a fully documented MIDI sequencer engine, able to manage songs, tracks This project dates back to 1996 and seems to be closed since 2003, but the sources are still available with Raspbian.

The QX1 schematics

The boards

The QX1 is architectured on the following boards:

Board DM: this is the main board, with
- the CPU (Hitachi HD68B09E)
- memory
  - RAM = 8 x MB8264-15 = 64 kB
  - ROM = HN4827128G-30 + 2 x HN482764 = 32 kB
- FDC (floppy disk controller, Toshiba MB8877A)
- 3 custom logic chips:
  - IC8 (called PMAC, ref YM5205), manages the FDC and serves as MMU
  - IC28 (called MASTER, ref YM5212), manages MIDI
  - IC31 (called PIT, YM5213), manages board PNA.

Board PNA: this is the I/O board, with

sub-CPU (IC6, Hitachi HD6803)
a ROM (IC5, MBM2764-30Z-G)
ADC (IC11, ADC0804)
Keyboard matrix (decoded via a pair of TC40H240P, IC1 and IC12)
Interface to LCD display
Interface to LED matrix
Interface to the Tempo potentiometer.

On this board, the address are:

Address	Description
0x0020	Data 7 = Foot switch Data 6 = Key "CLICK / Z" Data 5 = Key "TRNS / Y" Data 4 = Key "CHAIN / X" Data 3 = Key "PLAY" Data 2 = Key "REC" Data 1 = Key "EDIT" Data 0 = Key "UTLT"
0x0021	Data 7 = (not used) Data 6 = Key "STOP" Data 5 = Key "RUN" Data 4 = Key "⊳⊳" Data 3 = Key "⊳" Data 2 = Key "⊲" Data 1 = Key "⊲⊲" Data 0 = Key "REPT / W"
0x0032	Data 7 = (not used) Data 6 = (not used) Data 5 = (not used) Data 4 = (not used) Data 3 = (not used) Data 2 = (not used) Data 1 = Key "SHIFT ↑" Data 0 = Key "SHIFT ↓"
0x0033	Data 7 = Key "BEND / 8 / O" Data 6 = Key "PRGM / 7 / N" Data 5 = Key "CTRL / 6 / M" Data 4 = Key "TEMPO / 5 / L" Data 3 = Key "KBD / 4 / K" Data 2 = Key "+8va / 3 / J" Data 1 = Key "-8va / 2 / I" Data 0 = Key "INSERT / 1 / H"
0x0034	Data 7 = Key "fff" Data 6 = Key "ff" Data 5 = Key "f" Data 4 = Key "mf" Data 3 = Key "mp" Data 2 = Key "p" Data 1 = Key "pp" Data 0 = Key "ppp"
0x0035	Data 7 = (not used) Data 6 = (not used) Data 5 = Key "Bb" Data 4 = Key "Ab" Data 3 = Key "F#" Data 2 = Key "Eb" Data 1 = Key "C#" Data 0 = Key "SPACE"
0x0036	Data 7 = (not used) Data 6 = Key "B" Data 5 = Key "A" Data 4 = Key "G" Data 3 = Key "F" Data 2 = Key "E" Data 1 = Key "D" Data 0 = Key "C"
0x0037	Data 7 = Key "ENTER" Data 6 = Key "↑ / V" Data 5 = Key "↓ / U" Data 4 = Key "→ / T" Data 3 = Key "← / S" Data 2 = Key "JOB COMMAND / R" Data 1 = Key "0 / Q" Data 0 = Key "9 / P"

Board PNB: this is the 3 * 8 LED matrix board. Passive board.
Board PNC: this is the 2 LED board (RUN and TEMPO). Passive board.
Board JK: this is the MIDI, Foot switch and Tape board. Passive board.
The PSU board: provides +5V and +12V for the floppy drive.
The floppy drive: Canon MDD211, 5"1/4.

The connectors

CN1, connector to board PNA (Flat grey cable)

Pin	Description	Cable color
CN1-1	RSEL (Input)
CN1-2	Ground
CN1-3	STB (Input)
CN1-4	FSW (Foot Switch, Output)
CN1-5	CLK4 (Clock 4MHz, Output)
CN1-6	RES (Reset, Output)
CN1-7	EIRQ (External Interrupt Request, Output)
CN1-8	EXRW (External Read/Write selector, Input)
CN1-9	P1 (Data Bus 1, Bidirectional)
CN1-10	P2 (Data Bus 2, Bidirectional)
CN1-11	P3 (Data Bus 3, Bidirectional)
CN1-12	P4 (Data Bus 4, Bidirectional)
CN1-13	P5 (Data Bus 5, Bidirectional)
CN1-14	P6 (Data Bus 6, Bidirectional)
CN1-15	P7 (Data Bus 7, Bidirectional)
CN1-16	P8 (Data Bus 8, Bidirectional)

CN2, connector to board JK

Pin	Description	Cable color
CN2-1	RX+ (MIDI In +, Input)	Brown
CN2-2	RX- (MIDI In -, Input)	Red
CN2-3	THR (MIDI Thru, Output)	Orange
CN2-4	TX8 (MIDI Out 8, Output)	Yellow
CN2-5	TX7 (MIDI Out 7, Output)	Green
CN2-6	TX6 (MIDI Out 6, Output)	Blue
CN2-7	TX5 (MIDI Out 5, Output)	Violet
CN2-8	TX4 (MIDI Out 4, Output)	Grey
CN2-9	TX3 (MIDI Out 3, Output)	White
CN2-10	TX2 (MIDI Out 2, Output)	Black
CN2-11	TX1 (MIDI Out 1, Output)	Sky Blue
CN2-12	(not connected)

CN3, connector to board JK

Pin	Description	Cable color
CN3-1	AGND (Analog ground)	Black
CN3-2	TO (Tape Sync Output)	Brown
CN3-3	Vcc (+5V)	Red
CN3-4	MTR (Click Out, Output)	Green
CN3-5	(not connected)
CN3-6	TI (Tape Sync Input)	Red
CN3-7	Ground	Black
CN3-8	FSW (Foot Switch, Input)	Orange

CN4, connector to Power Supply Unit

Pin	Description	Cable color
CN4-1	(not connected)
CN4-2	Ground	Black
CN4-3	Ground	Black
CN4-4	Vcc (+5V)	Red
CN4-5	Vcc (+5V)	Red

CN5, connector to Floppy Disk Unit

Pin	Description	Cable color
CN5-1	Ground
CN5-2	HLD (Hold, Output)
CN5-3	Ground
CN5-4	(not connected)
CN5-5	Ground
CN5-6	SEL3
CN5-7	Ground
CN5-9	Ground
CN5-10	SEL0
CN5-11	Ground
CN5-12	SEL1
CN5-13	Ground
CN5-14	SEL2
CN5-15	Ground
CN5-16	MON
CN5-17	Ground
CN5-18	DRC (Output)
CN5-19	Ground
CN5-20	STEP (Output)
CN5-21	Ground
CN5-22	(not connected)
CN5-23	Ground
CN5-24	WG (Write Gate, Output)
CN5-25	Ground
CN5-26	TR0 (Track 0, Input)
CN5-27	Ground
CN5-28	PRT (Input)
CN5-29	Ground
CN5-30	RD
CN5-31	Ground
CN5-32	SIDE (Output)
CN5-33	Ground
CN5-34	RDY (Ready, Input)

TSE3

TSE3 Documentation

http://tse3.sourceforge.net/doc/doxygen/namespaceTSE3.html#a118

Compiling on the Raspberry Pi environment

Raspberry Pi (RPI) model B
Linux Raspbian 4.19.118+
GNU C++ Development Kit
TSE version 3-0.3.1, sources downloaded to folder /opt

What I did

Boot the RPI, go to session 1 (CTRL+ALT+F1)

Become root and build the project

$ sudo su
# date 070112552022  ← adjust accordingly...
# cd /opt/tse3-0.3.1
# ./configure --with-doc-install \
--without-oss --without-aRts --without-win32
# make clean
# make

I came into the following error:

make[4]: *** no rule to make target '../../src/tse3/libtse3.la', needed by 'test'. Stop.

I simply added the full path of target libtse3.la in Makefile:

# vi +522 src/tse3/Makefile
...
522 ${top_builddir}/src/tse3/libtse3.la: $(libtse3_la_OBJECTS) ...
...

I came into another issue during the compilation of FileBlockParser.cpp. The issue is actually in file Serializable.h:

../../src/tse3/Serializable.h: In function 
std::ostream& TSE3::operator<<(std::ostream&, const TSE3::Serializable::indent&);
../../src/tse3/Serializable.h:256:45: error: no match for operator<< 
(operand types are std::ostream {aka std::basic_ostream<char>} and const char [5])

This is actually a type mismatch in operand 2. The string must be casted to indent& on line 256 of Serializable.h:

# vi +256 src/tse3/Serializable.h
...
256  for (...) s << (Serializable::indent&) "     ";
...

With this change, FileBlockParser.cpp compiled successfully.

I got this error in TSE2MDL.cpp and MidiFile.cpp

error: strncmp was not declared in this scope
     if (strncmp(...)
         ^~~~~~~

Cstring.h has to be included:

# vi src/tse3/TSE2MDL.cpp src/tse3/MidiFile.cpp
...
#include <fstream>
#include <cstring>
...

Lastly, a similar error where Stdlib.h must be included:

# vi src/examples/recording/recording.cpp src/tse3play/tse3play.cpp
...
#include <stdlib.h>
...

The last step is to install TSE3:
```
# make install
```

Compiling on Ubuntu

My environment

HP XW6600
Ubuntu 20.04
The included GNU C++ Development Kit
Sources downloaded from tse3.sourceforge.net to folder /home/francois/Development/projets/QX1/tse3-0.3.1

Compilation

As my regular user:

$ cd tse3-0.3.1
$ ./configure --with-doc-install --without-oss --without-aRts --without-win32

I came into the following error:

...
checking for correct ltmain.sh version... grep: character class syntax is [[:space:]], not [:space:]
sed: character class syntax is [[:space:]], not [:space:]
no

*** Gentoo sanity check failed! ***
*** libtool.m4 and ltmain.sh have a version mismatch! ***
*** (libtool.m4 = 1.5.18, ltmain.sh = ) ***
...

The error is located in the configure script:

$ vi configure
...
 6344 gentoo_lt_version="1.5.18"
 6345 gentoo_ltmain_version=`grep '^[:space:]*VERSION=' $ltmain | sed -e 's|^[:space:]*VERSION=||'`
...

Fix the issue by changing these 2 lines:

 ...
 6344 gentoo_lt_version="\"2.4.6 Debian-2.4.6-14\""
 6345 gentoo_ltmain_version=`grep '^[[:space:]]*VERSION=' $ltmain | sed -e 's|^[[:space:]]*VERSION=||'`
 ...

Next,

$ make

yields to:

...
/bin/bash ../../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I. -I../../.. -I../../../src    -g -O2 -W -Wall -ansi -pedantic -c -o Demidify.lo Demidify.cpp
/bin/bash: ../../../depcomp: No such file or directory
make[4]: *** [Makefile:296: Demidify.lo] Error 127
make[4]: Leaving directory '/home/francois/Development/projets/QX1/tse3-0.3.1/src/tse3/util'
make[3]: *** [Makefile:481: all-recursive] Error 1
make[3]: Leaving directory '/home/francois/Development/projets/QX1/tse3-0.3.1/src/tse3'
make[2]: *** [Makefile:240: all-recursive] Error 1
make[2]: Leaving directory '/home/francois/Development/projets/QX1/tse3-0.3.1/src'
make[1]: *** [Makefile:315: all-recursive] Error 1
make[1]: Leaving directory '/home/francois/Development/projets/QX1/tse3-0.3.1'
make: *** [Makefile:227: all] Error 2

To fix, run:

$ autoreconf -vfi
$ make

Few hiccups will arise:

In file included from Phrase.cpp:17:
../../../src/tse3/cmd/Phrase.h:116:37: error: ‘TSE3::PhraseEdit’ has not been declared
  116 |                               TSE3::PhraseEdit  *phraseEdit,
      |                                     ^~~~~~~~~~

To fix this, add PhraseEdit.h to src/tse3/cmd/Phrase.h:

$ vi src/tse3/cmd/Phrase.h
...
 20 #include "tse3/cmd/Command.h"
 21 #include "tse3/Phrase.h"
 22 #include "tse3/DisplayParams.h"
 23 #include "tse3/PhraseEdit.h"
...

Some casting has to be added:

Song.cpp: In function ‘void TSE3::File::write(TSE3::File::XmlFileWriter&:, TSE3::Song&:)’:
Song.cpp:18:44: error: call of overloaded ‘element(const char [9], size_t)’ is ambiguous
   18 |     writer.element("NoTracks",  song.size());
      |                                            ^
In file included from Song.cpp:4:

Fix song.size as follow:

$ vi src/tse3/file/Song.cpp
...
 18     writer.element("NoTracks",  (unsigned int) song.size());
...

Do the same in Track.cpp:

$ vi src/tse3/file/Track.cpp
...
 16     writer.element("NoParts",  (unsigned int) t.size());
...

With these changes, the libraries compiles successfully.

However, there is one rule missing to build the library:

make[4]: Entering directory '/home/francois/Development/projets/QX1/tse3-0.3.1/src/tse3'
make[4]: *** No rule to make target '../../src/tse3/libtse3.la', needed by 'test'.  Stop.

The rule actually exists in src/tse3/Makefile.in

...
 440 test_LDADD = $(top_builddir)/src/tse3/libtse3.la
 ...
 522 libtse3.la: $(libtse3_la_OBJECTS) $(libtse3_la_DEPENDENCIES) $(EXTRA_libtse3_la_DEPENDENCIES) 
 523         $(AM_V_CXXLD)$(CXXLINK) -rpath $(libdir) $(libtse3_la_OBJECTS) $(libtse3_la_LIBADD) $(LIBS)

but the path is incorrect. I replaced the line:

 521 tse3 = $(top_builddir)/src/tse3
 522 $(tse3)/libtse3.la: $(libtse3_la_OBJECTS) $(libtse3_la_DEPENDENCIES) $(EXTRA_libtse3_la_DEPENDENCIES)
 523         $(AM_V_CXXLD)$(CXXLINK) -rpath $(libdir) $(libtse3_la_OBJECTS) $(libtse3_la_LIBADD) $(LIBS)

 In file included from ../../src/tse3/FileBlockParser.h:20,
                 from FileBlockParser.cpp:17:
../../src/tse3/Serializable.h: In function ‘std::ostream& TSE3::operator<<(std::ostream&, const TSE3::Serializable::indent&)’:
../../src/tse3/Serializable.h:256:45: error: no match for ‘operator<<’ (operand types are ‘std::ostream’ {aka ‘std::basic_ostream<char>’} and ‘const char [5]’)
  256 |         for (int n = 0; n < i.level; ++n) s << "    ";
      |                                           ~ ^~ ~~~~~~
      |                                           |    |
      |                                           |    const char [5]
      |                                           std::ostream {aka std::basic_ostream<char>}

To fix, iostream must be included:

$ vi src/tse3/Serializable.h
...
 20 #include <iosfwd>
 21 #include <iomanip>
 22 #include <cstddef>
 23 #include <iostream>

TSE2MDL.cpp: In member function ‘bool TSE3::TSE2MDL::load_header(std::istream&)’:
TSE2MDL.cpp:218:9: error: ‘strncmp’ was not declared in this scope
  218 |     if (strncmp(buffer, "TSEMDL  ", 8) != 0) throw std::exception();
      |         ^~~~~~~

Add cstring:

$ vi src/tse3/TSE2MDL.cpp
...
 40 #include <fstream>
 41 #include <cstring>

recording.cpp: In function ‘int main(int, char**)’:
recording.cpp:51:9: error: ‘exit’ was not declared in this scope
   51 |         exit(1);
      |         ^~~~

Fix with:

$ vi ./src/examples/recording/recording.cpp
...
 29 #include <iostream>
 30 #include <cstring>
 31 #include <cstdlib>

Also, cstdlib must be added to the following file:

$ vi src/tse3play/tse3play.cpp
...
 35 #include <cstdlib>

With these changes, TSE3 compiles with a couple of warnings, but no error.

Installation

$ sudo make install

Test

I created an alias for the newly built command midifile and tested it with a midifile I had on my computer:

$ alias midifile="`pwd`/src/examples/midifile/midifile"
$ midifile ~/Documents/MIDIFILE/35017.mid
[StreamMidiScheduler::ctor]     StreamMidiScheduler version 0.10 [dev].
[StreamMidiScheduler::tx]          0.00 - System.......... c:f p:0 d1:00
[StreamMidiScheduler::tx]          0.00 - System.......... c:0 p:0 d1:7e
[StreamMidiScheduler::tx]          0.00 - System.......... c:0 p:0 d1:7f
[StreamMidiScheduler::tx]          0.00 - System.......... c:0 p:0 d1:09
[StreamMidiScheduler::tx]          0.00 - System.......... c:0 p:0 d1:01
[StreamMidiScheduler::tx]          0.00 - System.......... c:7 p:0 d1:00
[StreamMidiScheduler::start]       0.-24
[StreamMidiScheduler::setTempo]    0.00 - 48
[StreamMidiScheduler::tx]          0.00 - Program Change.. c:0 p:0 d1:00
[StreamMidiScheduler::tx]          0.00 - Control Change.. c:0 p:0 d1:79 d2:00
[StreamMidiScheduler::tx]          0.00 - Control Change.. c:0 p:0 d1:0a d2:33
[StreamMidiScheduler::tx]          0.00 - Control Change.. c:0 p:0 d1:79 d2:00
[StreamMidiScheduler::tx]          0.00 - Control Change.. c:0 p:0 d1:0a d2:33
[StreamMidiScheduler::tx]          0.00 - Control Change.. c:0 p:0 d1:07 d2:5f
[StreamMidiScheduler::tx]          0.00 - Control Change.. c:0 p:0 d1:07 d2:5f
[StreamMidiScheduler::tx]          0.48 - Note On......... c:0 p:0 d1:41 d2:27  (F-5)
[StreamMidiScheduler::tx]          0.48 - Note On......... c:0 p:0 d1:44 d2:30  (G#-5)
[StreamMidiScheduler::tx]          0.48 - Note On......... c:0 p:0 d1:41 d2:30  (F-5)
...

Program a 8-channel bit banging UART

MIDI uses a simplified serial communication: fixed bitrate (31250 bit/second), 8 bits, 1 stop bit, no parity. Sending 1 bytes requires 10 bits, hence 3125 bytes max per second.

Boards

A STM32F104C8T6 Blue pill board
An official Arduino USB2Serial board
The Arduino IDE
A breadboard

There are 2 ways to program the STM32: via the USB2Serial board, or via the embedded mini-USB B. The Mini-USB is more convenient, but requires a specific bootloader to be flashed. For now, I will use the first method, as explained in this link. The second method is explained here.

Everything is setup according to the method, but so far, sending blink ends with an error:

Failed to init device.
stm32flash Arduino_STM32_0.9

http://github.com/rogerclarkmelbourne/arduino_stm32

Using Parser : Raw BINARY
Interface serial_posix: 115200 8E1

An error occurred while uploading the sketch

I tried with BOOT0 in DFU mode (position 1) or normal mode (position 0). To confuse things, my STM32 already contains a blink I may have send few years ago! So when I reset the board, it blinks! So, I will have to flash the bootloader to use the Mini-USB. More to follow...

Follow up
After hours of work, and reflashing the bootloader, I'm not able to upload via USB (the device is not recognized). I've read that there are many counterfeits, which could be the case for me as well (purchased 2$ at Aliexpress). And I changed my mind a little bit: I have a ATMega328P-PU with the Arduino bootloader (referenced A000048), purchased at Digikey - trusted source. I think this is a better option, since it is compatible with 5V, hence easier to interface with the other boards. More to follow...

Adventures with ATMega328P-PU

The ATMega328P-PU referenced A000048 is sold with the Arduino Uno bootloader pre-installed. I built a quick setup on a breadboard.

I wanter to upload Blink, from the Arduino IDE. However, I was not able to upload any sketch until I reflashed the bootloader. Once done, suprisingly enough, the flashing LED was the one on PB1 as expected, but PD0 - which corresponds to the LED TX on the USB2serial board. My guess is the predefined consts in the IDE correspond to the Arduino board, but not necessarily to the ATMega. After some twicking (select programmer: Arduino as ISP in the IDE, add a 0.1 µF capacitor on /RESET), I was able to blink 5 LEDs on the breadboard.

void setup() {
  DDRB = 0x3f;	// set port B[0:5] as output
}

void loop() {
  PORTB=0x02; delay(500);	// Set PORTB-1 as HIGH for 1/2 second
  PORTB=0x04; delay(500);	// Set PORTB-2 as HIGH for 1/2 second
  PORTB=0x08; delay(500);	// ...
  PORTB=0x10; delay(500);
  PORTB=0x20; delay(500);
  PORTB=0x00; delay(1000);	// Reset PORTB for 1 second
}

Next protoboard

I use 2 dual 7-segment + decimal point display referenced LTD432DP-YQ. Pinout is

Pin 4 is the common cathode for left digit, pin 5 is common cathode for right digit.

I'll use port C, bits 0-4 for left digit, and port D bits 4-7 for right digit. I will not use all the segments; only B, C, E and F.

ATMega pin 23 (port C-0) → display pin 13
ATMega pin 24 (port C-1) → display pin 1
ATMega pin 25 (port C-2) → display pin 2
ATMega pin 26 (port C-3) → display pin 14
ATMega pin 6 (port D-4) → display pin 12
ATMega pin 11 (port D-5) → display pin 8
ATMega pin 12 (port D-6) → display pin 7
ATMega pin 13 (port D-7) → display pin 11

Search This Blog

Lone in the night

Compiling TSE3 on Raspberry Pi

Goal and background

Identified steps

ALSA

TSE3

The QX1 schematics

The boards

The connectors

TSE3

TSE3 Documentation

Compiling on the Raspberry Pi environment

What I did

Compiling on Ubuntu

My environment

Compilation

Installation

Test

Program a 8-channel bit banging UART

Boards

Adventures with ATMega328P-PU

Next protoboard

Comments

Popular Posts

Where is Brian?

Emulation of Fujitsu MB8877 Floppy Disk Controler with Arduino