How To Control The WowWee Alive Elvis From An External PC (Part 3)

This is the third of four articles about adding a "parasitic" microcontroller to the WowWee Alive Elvis that allows control of this device from an external PC via a serial interface. This article covers the implementation of the interface’s serial commands to control the Alive Elvis bust from a MIDI music sequencer. You should not be afraid if you have very little or no knowledge of music to use a music sequencer for this type of application. Just keep an open mind that animation events are not much different than musical note events other than the music note events exercise your ears verses animation events exercise your eyes or more exactly your Elvis’ eyes, brow,mouth, neck, and upper lip. At the end of the details in this article, if you have used a music sequencer before you will be ready create your own animations for your Alive Elvis bust. Otherwise you will want to wait for the last article in the series which is about using a free demo version of a music sequencer (Multitrack Studio) to create some animations.

Reasons for using MIDI to control Elvis

The use of a MIDI interface for robotic control is not uncommon and there are many examples and at least one software package (VSA Servo Animation Software) that can be found on the internet. My particular reasons for using a music sequencer to control the Elvis Alive bust are mentioned below.

  • Any MIDI sequencer that supports audio tracks can be used, even free demo versions as long as they allow you to save your work.
  • MIDI note and MIDI Controller events can be used as animation control events and are definable down to a 2 millisecond level.
  • Sequencers support useful graphical editors for events and audio tracks. Graphical editors for the audio are useful to align mouth movements and adjust the size of the movement besides being useful for editing the audio track itself.
  • Looping of events is supported so that adjustments of the animation can be made in real-time, particularly the mouth.
  • Most have "piano scroll" editors for note editing which is an intuitive way for looking at notes as events and not as music (this is the same style as the VSA animation software mentioned earlier).
  • Sequencers support some customization so that animation events (notes) can be labeled on the note editors. IMPORTANT!!!! This feature minimizes your need for any knowledge of music and is not supported by all music sequencers.
  • The events can be saved into MIDI pattern files to build an "expression library".
  • Many sequencers support audio processing effects that can be used to process the audio track to process the voice audio for different effects.
  • Elvis means music with that voice and custom music can be created with the sequencer.
  • Completed MIDI songs (animation sequences) can be played back from a song playlist/jukebox option of the sequencer.
  • For a future project I am planning, a completed animation could be exported as separate MIDI and MP3 files and then converted by custom software into a format that can be read and processed directly by the microcontroller. This means the PC can be removed from the playback process and only used to play the animations on queues.

What is required to Animate PC Elvis

No new hardware for the PC is needed, as long as it already has a serial interface. However new software other than the MIDI sequencer will be needed. The software will be receiving the MIDI events that would normally go to a MIDI interface and it will send them to the RS-232 serial output to your PC Elvis bust. So all the software you need is as follows.

  • A MIDI sequencer (a freebee!). I am recommending Multitrack Studio Version 5.0 or 5.1 available here (http://www.multitrackstudio.com/setupmtslite.exe). This demo allows you to save your work and is limited to only three tracks but this is sufficient for this application. It also has a much simpler layout than many of the bigger sequencers (Sonar, Cubase to name two) but it still has all the needed features.
  • A custom program is required to receive the MIDI events from the sequencer and convert them into the commands to send to PC Elvis over the serial interface. I am supplying this one for Windows XP, another freebee with all the other PC Elvis files available here (http://www.mediafire.com/file/jmqtninhjzd/PCElvis.zip). You will need to be sure that no other program is using the serial port that this process is configured to connect to or it will terminate. I had to use another freebee call "Process Explorer" to find the rouge process that was using my serial port.
  • A freeware program (Hubi's Loopback Cable for Windows XP) to "connect" the sequencer and the custom program (http://www.nerds.de/data/setuploopbe1.exe). This program appears as a MIDI interface to the sequencer and to the custom program allowing both to connect to it, then talk to each other. (yet another freebee!)

Designing the MIDI implementation for control of PC Elvis

The RS-232 serial interface that was defined in the previous 2 articles uses the following commands to run the 10 motors of the bust.

  • RUN MOTOR at speed X for time Y
  • STOP MOTOR
  • GOTO POSITION Z at speed X

In addition there is a MUTE/UNMUTE AUDIO command which is useful when operating multiple busts. This allows a single audio channel to be directed to more than one bust as long as the busts do not need to speak simultaneously. Two busts could use the L/R stereo but why stop there?

There are many good web sites that provide a detailed definition of the MIDI interface specification such as this site (http://www.sonicspot.com/guide/midifiles.html). I will only touch on what it is needed here to describe how it is used for this application. The MIDI interface is essentially a serial interface running at over 30khz. There are serial commands sent over this interface for turning music notes on/off with an added data value that is used as the amplitude (velocity) of the note. There are a total of 128 different musical notes that can be played to cover over 10 music octaves. Additionally there are serial commands named Controller events for controlling all the characteristics of the sound of the notes as they are played back by the devices receiving the note on/off commands. These Controller events also have a single data value attached. All of these events can be routed to 16 different channels so you can command at least 16 different musical instruments from a sequencer.

To control the PC Elvis bust with MIDI, we simply need to map the Note On commands to become RUN, STOP and GOTO motor commands. We can use the amplitude information of these events to define either the position Z a motor is to move to (a GOTO command) or define the speed X at which a motor is to run at (a RUN command). The only parameter left undefined is time Y (for a RUN command) which is taken care of with the Note Off command as long as the motor is setup to run long enough after a Note On event until it receives the Note Off event.

In addition, we can use the MIDI controller commands to perform the same motor GOTO POSITION operation but not the RUN MOTOR since the MIDI Controller commands are not followed by a corresponding Note Off command. Also the controller commands can be used to control the MUTE/UNMUTE of the PC Elvis audio output.

Which MIDI channel (1-16) for the event also appears as part of the main identifying value for the most any type of MIDI event. So we can send all events to all busts and let each bust ignore all but its own events based on the Elvis Number we already have stored in its EEPROM via the microcontroller menu input over a serial terminal (see Part 2).

The MIDI equivalent to the PC Elvis Serial Commands

The following is the resulting mapping of MIDI events to PC Elvis serial commands.

  • MIDI Note On/Off to PC Elvis RUN MOTOR and STOP MOTOR – The RUN MOTOR will be useful for any of the spring loaded motors especially the jaw where subtle changes in amplitude and duration are required to match its movement with speech. Using a Note On event to perform a RUN MOTOR command with its single attached data value used to define the speed X is strait forward. The duration is not a problem since a MIDI Note Off event will always be sent and will use a STOP MOTOR command to turn the motor off. We just need to give the RUN MOTOR command a long enough RUN time to complete any expected note duration. This max run time value can be adjusted from the PC Elvis interface's test menu in the microcontroller (the default is set to 3 seconds).
  • MIDI Note On to PC Elvis GOTO POSITION. – The GOTO POSITION command is perfect to position motors to create the expressions while moving the mouth. The Note On events can also be used for this if we use the single data value that comes with the note to define the position to move to. However, the speed of the movement must be pulled from somewhere else because we have only one data value with the Note On event. Since changes in speed are something that will probably not occur as often as changes in position, the use of a separate Note On event can be used to define the speed of any individual motor with the value saved for all future GOTO operations for that motor until it is replaced by another speed value. Even a Note On event to set a global speed for all motors could be useful in some cases. For the notes used for the GOTO command, the Note Off event would always be automatically sent for these events by the sequencer, but will be ignored by our mapping program to allow the motor to reach its specified position.
  • MIDI Controller commands to PC Elvis GOTO POSITION - The GOTO position can also use a MIDI Controller command with the attached data value representing the position to go to and the speed pulled from some global speed setting that can also be set with a another MIDI Controller command.

So what does all this give us? It allows some flexibility in how you use the editors and graphical tools provided with the sequencer used to create animation sequences. Different sequencers have different strengths and weaknesses in their editors so this small bit of diversity in control will increase our options of which sequencers we can use.

Now all we need to do is map some of the 128 individual Note On/Off events and Controller events from the sequencer to RUN MOTOR, STOP MOTOR, GOTO POSITION commands for specific motors and send these commands out the RS-232 serial interface. No problem!

Actual MIDI Implementation for Note Control of PC Elvis

The tables below show the music notes that control PC Elvis. However, the sequencer will allow us to show these notes with more friendly names so remembering these is not required to produce animations. There are four octaves used as follows:

  • The notes of octave 0 control all the motors using note velocity as the destination position for GOTO commands and a single global speed setting that is applied to all motors except the spring loaded motors. The spring loaded motors of Neck, Jaw, and Brow are controlled with RUN commands that use the note velocity as the speed and the note duration to stop the motor (i.e. not stopped at a position except for the internal maximum limit).
  • The notes of octave 1 set individual motor speeds for the GOTO POSITION commands that are triggered using octave’s 2 notes defined below.
  • The notes of octave 2 are used exactly the same as octave 0 except the speeds for the GOTO commands are the independent values set with the notes in octave 1. Notes to control the spring loaded motors of Neck, Jaw, and Brow are duplicated in this octave for convenience and function the same as in octave 0.
  • The notes of octave 3 are used for independent control of the left and right eyelids either using the global speed or the independent speeds that are also set in this octave. All the eyelid commands are GOTO commands.

 

Octave 0

   
     
Motor Note Usage
     
HEAD Up/Dn C0 GOTO, positioned by note velocity, using global speed
SWING L/R C#0 GOTO, positioned by note velocity, using global speed

Comments

roschler's picture

Awesome job Evilis! :)

Currahee's picture

great!!!!

jofi's picture

awesome !!!