Elvis Animator Program

67 posts / 0 new
Last post
RetroPlayer
RetroPlayer's picture
Elvis Animator Program

I am starting this thread to discuss my upcoming Elvis Animator program. This program will let you create and edit custom animations for Elvis, which you can then put on your modified cartridge to playback on the Elvis Alive! Animatronic bust.

I am starting this thread because I am about half way through the major parts of the program, and am hoping to get some early feedback and suggestions for it.

The Source code can be shared with anyone that knows how to program in VB, but I will be the main distribution point. So, I expect that anyone that gets the source and modifies it, does not attempt to package it and deploy it separately (at least until the status is release candidate, and then you can do whatever you want with the source code.)

The program is being written in Visual Basic 6.0, which is a fairly obsolete language, but it all that I am familiar with right now for windows programming, except a little Delphi, for which I no longer have an IDE.

I would definitely like some involvement from others seeking to port the program to other languages and working to make it as open platform as possible. This is FAR FAR beyond me, so without help, only a basic windows-based program will be available.

-------------------------------------------------------------------------------------------------------------------------

This section will be updated with the status of the project

-------------------------------------------------------------------------------------------------------------------------

Status: Alpha

RetroPlayer
RetroPlayer's picture

I am looking for alpha testers. Right now, what I am looking for is feedback on the usability of the program and suggestions for features or improvements.

Being an alpha tester means that you will need to be somewhat proficient in the use of Windows.

An alpha tester will install the program with example animations and sound files and will basically just play with working features. They will provide me with frequent feedback on their overall feeling of the program.

You do not need to have a modified cartridge, or even Elvis for that matter. At this point, you are only working on files.

Preferred would be someone familiar with looking at files in a hex editor to examine the output of the files.

The program installs and runs fine on my XP system without crashing, but it is not installing properly on Sevik's XP machine. I am looking for answers to this, but you should expect that it might give you problems as well.

Most importantly: DO NOT EXPECT A FULLY FUNCTIONING PROGRAM YET

If you are not comfortable with that and really just are impatient to try it, then I really don't recommend getting involved at this point. It is not at a point that will be very useful to you yet.

So, if you are interested in taking a look at it and playing around with it and are willing to provide me with frequent feedback, PM me with your email address. I do not want to post it to the regular downloads section yet, because it is not ready to get put out in the wild.

Thanks in advance and I really appreciate it!

Retroplayer

RetroPlayer
RetroPlayer's picture

This is the current state of the program. The pretty little graph display is displaying the path of the selected motor (Eyes U/D) throughout the entire animation. This will be displayed below the waveform display (when I finally get that working) and will give you a good idea of what the motors are doing during points in the audio.

The wierd little box in the upper left, is the beginnings of a "Simulator" which will allow you to play back the animation and watch the movements on an animated head. This is going to be pretty complex, but so far it's been easier than figuring out how to display the waveform. :)  If I still have some energy when I am done with this program, I may work on changing the simulator over to a 3D model and animating that. Also, another "maybe" that I am interested in attempting, is to allow the program to load profiles, which convert it over to another animatronic. I will decide this when I see the final look and complexity of code already involved. Making it configurable like this means essentially building the forms at runtime, which is not a simple thing to do :)

I will put a full size screenshot in the gallery.

RetroPlayer
RetroPlayer's picture

I now have a "spectrum" display running in the main form. But, currently it is not formatted the way it will be when finished. However, it does allow you to see the waveform frequencies while playing back. Also, the routines to display this will be needed for lipsyncing routines later.

I have also added a toolbar and status bar to the bottom. To process the audio, I need to freeze the form and play back the audio to get all the samples from the audio. I am working on trying to make the playback rate 2 to 4 times faster than normal so that you do not have to wait the entire length of the audio.

Anyway, this will only happen once at load time, or if you change the audio clip. Then all needed information will be in memory.

Also, with the new spectrum routines, the sliders are starting to lose track towards the end. But, once all this is loaded in memory, it will be much much faster.

Here's a screenshot as of 10 minutes ago:

 

Still no PMs from anyone offering to test this out. I could really use some feedback as it is difficult for me to remember to try everything. :) But, again, if you are not able or willing to deal with a little frustration, then now is still not a good time to get involved.

However, any feedback or suggestions from just looking at the screenshots would also be helpful.

RetroPlayer
RetroPlayer's picture

Currently Working:

Open Existing animation: grab associated audio clip, and fill all form controls

Create New Animation: This can be done by opening an audio file, and it will get the duration from that, or you can choose a duration. You can choose a duration longer or shorter than the audio clip after you load it.

Save Animation: This will overwrite the opened animation with your modified values

Save As: Save modified animation or new animation to a new file

Frame by frame modification: As you scroll the bar in the middle, the sliders will update to the positions of the movements in that frame. You simply move the slider (and the path display will update as well) to your desired position for that frame. The motor speeds will be calculated when you save the animation, based on how much change there is between the previous and current frame. Sevik suggested allowing this to be configured in the options.

Note on File access: The program will deal with frames that have less than 12 tracks by pre-filling the next track with the previous track's values. This is how idle is done in the existing animations. When saving, all frames will always be 12 tracks.

Play back: This will play the MP3 and also move the sliders in step with the animation. Once the simulator is completed, you will be able to watch an animated version of the bust during playback

Stop: Stop the playback of animation and MP3

Pause: Pause the playback of the animation and MP3

Motor Path display: This shows the activity of the selected motor over the entire animation. I am working on modifying this to display all motors at the same time. I made this display at first to calculate the speeds (by using slope) and it was originally only for debug, but it really seems to be useful so I will be improving on it.

Spectrum Display: This is basically a graphical EQ, which shows the amplitude of frequencies during the current sample. So, the bars climb and fall during playback. This will be modified to build a histogram of the entire sound file, with the voice bands set in white throughout the center. It is difficult to explain this, but basically what it will do is show the volume of the voice based on the 'energy' of the sound. This is probably a better indicator of jaw position. I will allow for the option to switch this display with a normal waveform display.

Random Fill: This is a really basic routine that allows you to choose three different movement groups to randomlt fill. You can fill a new (or existing) animation with blinks, eyebrows, and eye movement. All are working, but not as good as I would like. You can press fill as many times as you like to get the animation to look how you want. The motor path display will update each time you do a fill to show you the results.

Todo:

Finish spectrum and waveform displays
Finish Simulator
Finish LipSync routine

Routines for importing motion capture formats
Routine for importing plain text scripts
Routine for exporting to different formats
Look into providing the ability to configure a joystick to control the motor movements.

Add some framework for outputing the animation for real-time control. Though, if I get that far, I think I will have a separate program to do that.

Add frame editing functions, like insert, delete, copy, and paste

Miscellaneous aesthetics and functionality

Clean up and error trap my code. Make it cleaner and slimmer.

RetroPlayer
RetroPlayer's picture

Oh, You can also choose not to open an MP3 to associate with a new animation, if you like. I may add the ability to add an audio file separately.

GWJax
GWJax's picture

Retro, Ok I opened your source code and I have an error on the Form1 file and the Anamate form file, I think that is what it is called, There must have been a problem when uploading it to the site, I can read all of your code and run the program but with those errors I believe I'm missing some parts. Let me know if you can reload your new version source code so I can start helping you.
Jax

RetroPlayer
RetroPlayer's picture

Is it maybe complaining about Fmod? You will probably need to copy fmod.dll to your windows system32 folder.

Other dependencies (that I know of) are windows common dialog control, and windows common controls SP 5 and 6.

All depencies files are in the output/support folder.

The output folder contains an installer, which will copy all the dependencies to the proper places and register them. This is probably the easiest way to make sure that you have all the parts in the right places.

Anyway, so much has changed since the package I gave you on Friday, that you really should get an update anyway. I will package it up for you in the morning and get it posted.

Just to let you know, though, I have decided (actually tonight, just before leaving for work) to completely rebuild the program. So much of it was designed by trial and error (I had even forgotten how to open a file in VB when I started on this.) The code is an absolute mess with fragments all over the place and some really odd (but working) program flow. But, mainly, it is just getting to be almost impossible to maintain. Every time I make a small change, I have to update the code in 8 or 9 different places. And this is just bad coding practice. Also, I have practically no error trapping in the code, so it will just go nuts, leak memory, and crash if something gets off on the wrong path or if you do something I hadn't planned for. This is an absolute must if I am going to release the program to the public.

Now that I am far enough along and all major and most important functions are working and *mostly* stable, I should be able to rewrite all the routines that I am happy with cleaner and slimmer and freeze these parts.

Regardless, the new code will be built off mostly the original routines, so it is still useful to check out.

I'll update the package in the morning and let you know when it is ready.

RetroPlayer
RetroPlayer's picture

Currahee said: nice

Thanks, Currahee. This program is going to be pretty sweet when it is done. I have some big ideas for some things that I probably won't be able to do in the first version, or ever, but I am moving along so quickly on it that I am energized to keep working on them until I have all the features I want.

I am probably getting close to being able to release a beta soon. However, I really need some people to run through the motions and let me know if they can get it to run on their machines. So far Sevik has had trouble and now GWJax.

I will probably try to run it on a non-development machine in the next few days to see if it gives me trouble there and hopefully figure out why the only two people that have offered to help, so far are having trouble even getting it to run on their machines.

RetroPlayer
RetroPlayer's picture

I have pretty much gotten my old code ported over to a new project. I have managed to clean up my code quite a bit and slimmed everything down so it will run much faster (The playback works like a charm now.) I have also gotten rid of a few quirks that had been driving me nuts and ended up on my todo list.

This required me to go through every single procedure and line of code and determine if there was a more clear or more simple way to do it. It's still not perfect, but I am satisfied for now.

So, here are some updates:

1. I am implementing a project methodology to the program. This means, that you will be able to load or create several animations in one single sesssion and store them in your project. Since the files on the cartridge belong to 4 different categories, this made sense. You will be able to save and load a project, as well as "build" it to an output folder (such as an xD card in your reader) Building the project will name all of the files properly and collect them from whereever you have them stored.

2. I have mostly worked out the waveform and spectrum displays. I will be allowing the ability to switch between them at any time.

3. I am moving to image toolbars for the menu system including some custom icons that I made

4. Random fill routines still need lots of improvement

5. Frame routines like insert, delete, copy, paste and trim still need to be done. Should just be simple math routines

6. I am really not happy with the slider interface and am developing a much better interface. Still working on it though, and I am not 100% sure which way it will go, so it will probably end up in version 2, not version 1.

7. I have not worked on the simulator, or lipsyncing features yet. The lipsyncing, I think should be pretty easy once I get to it, because we are only modifying one or two motors (Lip and Jaw.) What I will need to calculate, I think, is just the amplitude and energy of certain frequencies. I will be getting that data to plot the spectrum anyway.

8. Some big features that are planned are joystick control, importing and exporting of motion capture formats, and real-time PC control (once I get the custom firmware working.) I will be outputting this to a serial port, so others could use it for their own controllers (GWJax) if they just implement the handler in their microcontroller. I am not sure which, if any of these features will make it into the first version.

I don't think anyone is really in a hurry since I have yet to hear about anyone with a custom cartridge besides me yet, but I would like to get the first version out there so I can get some people to provide feedback. I am a little disappointed that I haven't gotten any more offers to try it out. I know making a cartridge is tough for most people, but this is really just running a program... no cartridge needed. :) It might not run for you at first, or it might crash when you try to do something I didn't expect you to try, but it will not eat your operating system when it does. Well, I can't promise that, I guess, but it is very highly doubtful.

RetroPlayer
RetroPlayer's picture

And of course, a fresh screenshot. Sorry, having fully ported over the waveform display yet, and some of the buttons still need icons made:

RetroPlayer
RetroPlayer's picture

I just had to do some calculations and for cartridge filled with 100 animations for each category (completely maxed out) just the animation files alone would be 1.8GB. This would last for nearly 15 days, so obviously the size of the matching MP3 files would be very very large.

Of course, the max limit for FAT16, the format used by the cartridge, is 2GB, so really, you could never reach this limit.

So 600mb of just the animation script files for one maxed out category, with max duration.

Honestly, though, I cannot imagine someone programming an single animation that would last for 4 hours :) Just thought people might find the numbers interesting

RetroPlayer
RetroPlayer's picture

RetroPlayer
RetroPlayer's picture

Let's see if I can describe the picture above...

What you are looking at is a concept image for a manual motion capture system.
The idea is that you record some video of your own face acting however you want Elvis to act.

You start with a nuetral pose, which you use to place the markers (white dots) at the features on the face.

You line up your centerline, leftline, and rightline with your face so it has an idea of the scale of your face.

After you get all the markers placed, you press step and this will advance the video approximately 200ms (1 frame) Then you move the markers again to match the position of your face in that frame, and so on throughout your video.

The markers (white dots) are measured in relationship to each other to determine the expressions of the face. All of those lines you see are linked in a hierarchical stucture in relation to root (Red dot)

For example when you nod your head, the distance between then chin and root will shorten, but the position between nose and chin will not. When you open your mouth, the distance between nose and chin will increase but not nose to root, and so forth.

When you turn your neck, the angles of the link lines will change as well as all of the markers in relation to the center,left,and right lines.

It's not perfect, but I have been trying to develop a concept for simple motion tracking. There doesn't seem to be any free programs to do this, and even the ones you can buy are very complex. The facial movements of Elvis are very limited so a simpler approach was needed.

Really, when this is finished, the framework of it could be used for automatic optical motion capture, if you put real markers on your face.

The overall idea, is to use the capture as keyframes and then process this data to derive tweens (transistion frames)

The picture above is just a prototype UI designed in a paint program, so don't be misled into thinking I have it all working yet :)

Even though this is much simpler than commercial systems out there, it is still pretty complex. But, really this method is on scale with the simulator (and many of the routines will probably be shared between them) so it is at least feasible with alot of work.

Nocturnal
Nocturnal's picture

That just looks darn creepy.

RetroPlayer
RetroPlayer's picture

You don't care for my bald Elvis, Nocturnal? :)

Nocturnal
Nocturnal's picture

Only in a sort of horror movie, oh my god its going to kill us all, kinda way.

GWJax
GWJax's picture

Well ncturnal you wanted to rule the Earth, mabe this will help out alot in getting there.

RetroPlayer
RetroPlayer's picture

I should put some piercings in his face ;)

Then it will look like a mug shot with biometric data

MrScott
MrScott's picture

You've got a good start on the gunfighter robot from Westworld, as played by Yule Brennar.

 

RetroPlayer
RetroPlayer's picture

 It has taken me forever to get this working, but finally, I see some success! :) (Thanks for working through it with me, Sevik!)

It still isn't perfect, but at least I am plotting actual samples from the MP3 to the display. I need to tweak it a bit and it will get slapped into my main code. Right now, you are looking at just a prototype form that only serves the purpose and loading and plotting the MP3.

RetroPlayer
RetroPlayer's picture

Now, we are talking! This is a comparison of my display versus the display of the same MP3 in audactiy. We are getting there. I can certainly make it prettier, but at least I have the basics down now. Each of the red lines in my display is *supposed* to represent 200ms of time. Why 200ms? Because that is how long one frame of the animation lasts. The final display will scroll a bar along the waveform to show you where you are at in the animation and will highlight that section of the waveform. So, the red lines are mainly just there now for debugging to be sure I am seeing the entire waveform and not scaling it funny. The previous picture was only displaying the first half of the waveform, but now it is all displayed, and it is pretty fast!

When I am finished with this code, I will probably post just the display code somewhere, because I have found hundreds of people trying to figure out how to do this, but nobody actually had any answers for them other than *theory*.

The next step is to build a proper spectrum display of the entire MP3 so that you can switch between the two. I also need to reduce the number of samples to display so it is faster and prettier. This will require a routine to interpolate the wave form data to only display large changes in values throughtout each 200ms "chunk". I have a general idea how to do it, but need to nail some numbers down to build a proper algorythm.

RetroPlayer
RetroPlayer's picture

Thought you guys might like a video of the program in action. This is just playing back an animation.

RetroPlayer
RetroPlayer's picture

Here is another one demonstrating the editing of frames.

GWJax
GWJax's picture

Nice work retro!!!

RetroPlayer
RetroPlayer's picture

Thanks, Jax. I am trying to finish up some of the frame editing code (like copy, paste, insert, etc...) and then I will point you at the newest source code.

It's a good thing that the video quality is so lousy or it would have given away an easter egg I am planning:)

GWJax
GWJax's picture

Cool I love easter eggs!!

RetroPlayer
RetroPlayer's picture

I think, you, in particular will get a kick out of it when you find it :) I'll have to remember to remove it from the source code before giving you an updated copy. :)

RetroPlayer
RetroPlayer's picture

I think I am beginning to run out of steam a little. I am not coding as fast as I was, but I have spent pretty much all of my free time on this for the last two weeks at least, so I guess that's understandable. After my last post yesterday, pretty much all I did was some general code cleanup and fixed some bugs in the appearance that I noticed when looking at the video again.

This latest one is technically "Version 4 alpha" because I have restarted the project 4 times now. :) I think for now, I am happy enough with this one to stick to it until a release. Then I will incorporate the more advanced features I am planning later.

One of the things I have started to do that helps me prevent making things unmanageable is to build a completely separate project for each new feature that I am trying out and then just fit the final code from that into my main project. Before, I was testing everything out right in my main project, and wreaking havoc on it in the proccess.

RetroPlayer
RetroPlayer's picture

BTW, Jax, I will be adding in some real-time serial output from the editor at some point. It would help to define that ahead of time, so when you make your custom controller, for example, my animator program can communicate with it.

This way, you could use the official animation format and still play them back with your controller. I am assuming that you intend to communicate with your PIC from the serial port of a PC, right? When I modify the firmware to accept serial commands, I will also make it compliant with what we come up with. Then, either way someone wants to build it, they can use the same format and programs.

RetroPlayer
RetroPlayer's picture

So, I started writing a mini-app to play the animation out through a serial port by formatting it for servo controllers. I started looking for user manuals for the various servo controllers to add them, and GOOD LORD, there are tons and tons of them and the closest thing to a standard is the MINI-SSC. The Mini-SSC is very outdated though, and has no proportional speed control (this is done by sending the servo signals at faster than 50Hz) Not a huge deal, but I bet it isn't really used much by many robot enthusiasts.

So... I am hoping someone is actually reading my threads and can pipe in with what brands of servo controllers they have used or just think is really popular. I have to narrow it down, somehow :)

Right now, I have the manuals for the Mini-SSC series, the Lynxmotion (my fav), polulu, pontech, and basicX. Any other really popular controllers out there that should be included?

Pages