It’s alive!! – Well almost…

Last night I finished off some of the more complicated parts of Ethestra, like all the quantisation techniques for all the note length, position, velocity and pitch. The pitch area still needs some work, as I’m planning to make it support multiple chords, to do so requires pulling in some data from an old friend of mine called Tigla.

Tigla was an app I tried to write a while back to allow me to play guitar chords on my Nokia N810, and for them to sound realistic. At the heart it was just a sample player, using 6 wav files with offsets for each note, but it was also something else. I had written a fairly sophisticated chord library tool, which could take the name Em, AM, A7, and return a list of allowed notes in that scale. Well, the idea is to reuse that with Ethestra.

Fear not, though, here is the first rendering of Ethestra v0.65. It will be very interesting to see how v0.7 turns out 🙂

Ethestra Update

So, the project has been continuing nicely recently. I decided it would be awesome, if the end user, me, could add filters to the traffic for each instrument. I only wanted a single scapy instance sniffing for traffic, so I had to implement the filter in Ethestra, as opposed to sing scapys built-in filter system.

Thsi left me with a problem. I had no idea how to write a parser. After thinking about it for a while, and talking to some coder friends, it became clear that if I could get the filter in the format below, I could evaluate it fairly easily.

[["ip", "==", ""], "AND", ["sport", "==", "25"]]

This turned out to be true. With a little help from pyparsing, I can now enter filters like this.

ip == AND sport == 25

Listing expected soon…

Taking the project one step further

Some of you will have read my post yesterday about my new pet project, now code named “Ethestra”, a combination of Ethernet and Orchestra. What!! All the good names were taken!!

I put the system on in the background whilst I worked for an entire morning, and I have to say, I felt very relaxed. It was kind of an ambient/zen type feeling. Very soothing.

So I got to thinking how to take the project further. Is there anywhere we can take this? Of course. As I sit at my keyboard, I have just spent the last 2 hours, and now have a fully functional tracker type midi sequencer in python. It has no GUI of course. A simple drum rhythm is defined as a pattern variable, like so.

(0, 0x48, 0x60, 4),
(2, 0x48, 0x60, 4),
(8, 0x48, 0x60, 4),

(16, 0x4B, 0x60, 4),

(0, 0x4E, 0x60, 4),
(8, 0x4E, 0x60, 4),
(16, 0x4E, 0x60, 4),
(24, 0x4E, 0x60, 4),
(32, 0x4E, 0x60, 4),
(40, 0x4E, 0x60, 4),
(48, 0x4E, 0x60, 4),
(56, 0x4E, 0x60, 4),

Ok, so it’s not pretty, but the ultimate goal of the project is for the network data itself to create the patterns. That being said, it can create at least something musical. A sample of the sequencer alone is here (this is not network music yet).

Listening to my network traffic

I know this is nothing new, and that people have done it before, but I wondered what it would sound like if I hooked up my network interface to my speakers. Not, not quite that low level, but using scapy, an awesome python network toolkit, and python bindings for portmidi, I was able to hook the two up. I then piped the midi to ZynAddSubFx for the audio creation.

Now, I hope you’re not expecting a symphony, cos you ain’t gonna get one. What you do get is a kind of ethereal sound, with a sudden flurry of activity. I apologise for the slight pops and such, I need to tweak audio settings and run it on a faster CPU, but you get the idea. The plinky plonky noise is www data. The low level AAHs that you only hear occasionally are DNS calls.

About 20 seconds into it, you’ll hear some drums kick in, that’s ICMP data there that is. I set a ping going to give a rhythm. It’s all very, spacial and weird. At the moment most of the instruments cycle up a very small arpeggio. One packet turns the note on, another turns it off. It starts at one note, then the next, jumps up, then jumps up again, before going back to the original position.

I would be interested to hear what you think. There is also another instrument in there doing my jabber conversations. Heheheh. Enjoy.

Took about 2 hours from scratch to code and figure out how to use scapy and portmidi. Quite fun. Once the code is in a better state, I’ll probably even post it up.

The file is here

LOST Finale : A review from a die hard fan

So, I watched the finale of LOST last night. Did it answer everything? Most certainly not. Did it answer enough to pass as an ending? Probably. It seems to me that the writers decided, as many people have stated, that there simply could not be a logical explanation for everything. Bear it mind that logical, does not have to equal credible. We ended up with a kind of supernatural ending, which in a way I was kind of expecting anyway.

There were so many aspects of the show that were mysterious, so many ‘what the heck’ moments, but in the end, most of these were swept under the carpet and we were left with a rather limp ending, which in all honesty could have been tacked onto the end of each season and still made sense. There was literally no need to introduce the numbers, or time travel or any of the other wacky and weird things which LOST was known for. Of course they enhanced the journey, but that wasn’t the point. The writers seemed to be egging us on, making us think that in the end, all of these totally barmy ideas, would be boiled down into one single “ah-ha” moment.

It was resolutions like “You became a mother”, which Jacob tells us was the reason for Kate’s name being scratched off the wall, which just seemed so simple and yet so right, that made me love the show. Little subtle hints like this seemed abnormally absent. There were no real answers in the finale. When they began to bring in the Daniel Faraday storyline, with the experiments and physics, I became a little more excited. Finally there was a glimmer of hope that the whole premise of the show could be explained in someway with a scientific background.

Alas, none of this was explained. It felt as if the ending could have been applied to the end of any of the seasons and the show would have been completed just fine. It was like a failsafe, seemingly conceived from the beginning, which rendered everything that had happened in between just moot. Things just didn’t matter. The Dhama, didn’t matter. The numbers, didn’t matter, though there were some hints to the origin of these in some extra material. Shoddy work, in my opinion, it would have been much better to include the origins of these numbers in the main show.

Everyone knew there was never going to be a full explanation for everything. We had to take it for granted that the electromagnetic energy had strange effects, (remember the “box” people?), but at least give us something as a parting gift. Something which shows us that everything had a purpose. Something that explained some of the origins that were made so much of during the seasons and faded away into the ether in the finale. From a spiritual point of view, yah, they probably didn’t matter. The journey is what was important, but for all those people who won’t or can’t feel that way about characters, it felt unfair to leave them with nothing more than a bad taste in their mouths at the end of the this 6 year long journey.

Shortly before the end, my wife and I were discussing possible endings and I have to say I think she came up with a much better ending than the one we were given. Though it may have been slightly more predictable, (well we thought of it didn’t we?) it seemed more LOST like. There would ultimately be some sort of struggle at the end, as we saw, but then all the main characters would flee the island, leaving Jack and Locke sitting together on a beach, in much the same way that we saw Jacob and Smokey earlier in the season. It would have been funny also to supplement this with a plane flying overhead/ship wrecking on the rocks, and the pair looking at each other. It would have lent to the cyclic nature of the show which seemed so prevalent.

Overall, the ending was acceptable, but by no means good, or even great. It’s such a shame that the writers didn’t take the opportunity to really go all out on this and give us an ending we’ll never forget (for a good reason). We were quite keen to buy all the seasons on DVD once the finale was over. Now, I don’t think we’ll bother. Disappointing people, very disappointing.

Gaming : (N)othing new (AT) (AL)l??

Have Microsoft broken all the boundaries? As I was perusing the net the other day, I came across a video on the BBC website, which was showing an application of project Natal, that Microsoft had been demonstrating at E3. The introduction to the video was claiming that this was something pretty special and I’ll have to be honest at first glance, it certainly did seem a little too good. I thought it would be interesting to take a look at the video and analyse it a little. For those of you that haven’t seen it, or indeed can’t, I’ll give a short text description here.

The video starts with a woman walking up to a screen and greeting a small child who was playing on a swing. He walks over and greets her back. They then enter a discussion where the woman, Claire, questions the boy, Milo, as to whether he has done his homework or not. The boy then changes emotion, putting his head down and starts walking with shoulders hunched, not looking at Claire at all. The narrator points this out and describes a technology where Milo can recognise Claires emotions and visa versa. Interesting. As we continue, Claire offers to help Milo with his homework. He throws her a pair of goggles, which obviously can’t permeate through the screen into the real world, but Claire stoops to pick up the virtual goggles. He tells her to put the glasses on, and she uses her hands to make goggles like shapes in front of her eyes. Milo acknowledges this, and the camera then shifts to look into a pool of water, where Claire is now able to interact, by waving her hands in front of the screen, to make small waves in the water. After this she decides to help Milo and draws him an orange fish on a piece of paper. She shows this to a device above the screen and Milo reaches up and grabs what appears to be a copy of the drawing from above the screen. We hear him exclaim that it is orange shortly before the video finishes.

Clever stuff I hear you cry. Well yes and no, I feel that in some sense the video may be misrepresenting what is actually going on in front of our eyes. Now don’t get me wrong, the Natal framework certainly looks impressive, but I wanted to take a look at current technologies and see whether there is actually anything new in this at all. First of all we have facial recognition, Milo clearly recognises Claire and responds to her by name. Though facial recognition hasn’t been perfected, many machines are able to tell the difference between several faces. Head tracking and face tracking is something that even digital cameras can do nowadays and so this doesn’t surprise me. To be honest, let’s look at the market for this framework. It’s largely going to be of home entertainment use. Owing to that fact, the number of faces it has to differentiate between is likely to be small, often consisting of two adults of differing gender along with two children separated by age with a few years. I’ll admit I’m stereotyping a little here, but it’s nothing to be concerned about, any family is going to have similar differentiations between the various occupants.

Moving on from this we have the voice recognition. Voice recognition hasn’t received a huge boost to it’s technology of late, but it’s still good enough for recognising a few keywords. Extending this to the Natal framework and it’s hard to see whether the conversation is free form or scripted. Listening to the narrator speak about the project, and watching a few things on the screen it concerns me that the video is little more than a glorified script. What makes me say this? The fact that the narrator explains that everytime the pair of goggles is thrown at the interactee, they stoop down to pick them up. This seems to me to indicate that events are not at all free flowing and still have to utilise a large amount of pre-scripted effort. This is further confirmed by the feint but still visible symbol on screen of how to make the goggles symbol and this is repeated at the beginning of the demonstration where it appears Claire has been prompted to wave to Milo. It seems the NATAL system is driven by gestures and symbols. What did intrigue me is as Milo skips off to the pond, he mentions in conversation “I don’t know until I try do I?” This seemed to be a rather out of the blue sentence and could indicate more realism to the whole system, or a string of random phrases that Milo may utter, after discussing homework.

The emotional state of Milo is something which is touted by the narrator quite heavily in this video. He claims that Milo is able to recognise emotions in the interactee and is also able to exhibit emotions back. The second claim is a little easier to stomach. It’s entirely possible to put modifiers on the motion sequences to make them look happy or sad. Dropping the head, slouching forward is nothing special. The former of the two claims is more difficult to stomach. Just how can Milo recognise emotions from the interactee. In the video, we do not actually see any evidence of this, but it could possibly be achieved by monitoring the persons own stance and features of voice. Milo’s voice does indeed seem to change with his emotion, lending his voice to vary considerably depending on his “emotion”. This could be achieved quite easily with having a number of responses, dependent on the input of the interactee. Some could be happy, sad, surprised and based on keywords from the voice recognition and emotion analysis from stance and possibly face.

The next subject is one which unless the system is really limited I can’t fully explain. The synthesis of speech is actually really good. Along with speech recognition this appears to be an area which has been lacking in technological development in recent years. It could be that the demonstration has pre-scripted lines which Milo can speak, or it could be that the words can be generated on the fly. The NATAL sensor is apparently equipped with a multi array microphone which enables it to do acoustic source localisation and noise suppression which could aid the speech recognition, but the speech synthesis would probably be handled by the software on the console.

Next comes the interaction with water. Now in my mind, this is the easiest portion of the demonstration. There are a few nice touches, but again there is nothing ground breaking here. The sensor in NATAL is apparently capable of doing 3D full body motion capture of up to 4 people. Taking the movements of the Claire and making her ripple the water really is child’s play. It was, however, refreshing to see her reflection in the water. Presumably the RGB camera in the sensor is used to map video onto a plane which is then “rippled.” To be honest though not technically impressive this was one of my favourite parts of the demonstration video. The camera is also used to take a quick photo when Claire draws a picture of a fish for Milo. Though we hear Milo exclaim that it’s orange, the video ends before we can see whether he recognises it as a fish or not. Assuming that Milo is expecting to see a certain set of shapes, it isn’t beyond the realms of possibility for the software to be able to pick out rudimentary shapes from the drawing and convert those for Milo to process.

Some of you reading this, who have watched the video may be thinking that I’m being a little harsh and that the video was pretty amazing. I’m not denying the fact that the video was impressive at all. However after my first initial watch I decided that I wanted to dig a little deeper, and not take everything on face value. I wanted to see whether Microsoft were bringing anything ground breaking to the market. In my personal opinion the technologies behind this are nothing new at all. What NATAL does appear to bring, is a way to amalgamate all of these new technologies together into a single package. If the API behind this is as good the demonstration video, then it will be very interesting to see what the XBox360 has to offer, once NATAL is released. To be honest it is all going to hinge on what Microsoft do with the technology. Having a great technical demo is one thing, but being able to turn that into an immersive gaming experience is a completely different thing altogether. After all, we all have virtual reality now don’t we? Oh….yeh….what did happen to that?

Humour : The wonderful wizard of letter writing

Few could argue that our lives haven’t been bettered by the introduction of our favorite pointy hatted friend, the Wizard. So let’s take a look at the world without the virtual sorcerer.

Dave was sat at his desk. He’d been mulling over the problem for a while now, but he just couldn’t quite get it right. Top? Bottom? It just didn’t make sense. The middle would make it look awful. Eventually after much huffing and puffing he sat bolt upright and called across the office. “Wizard!! Oi Wizard.” Nothing. It was time for something a little more drastic. Dave flung a stapler over a partition and shouted, “OI…MR POINTY” A rather strangely dressed man appeared on a wheely chair and hurled himself towards Dave’s desk. He spun the chair as he went hoping to impress or at least announce his arrival.

“Tada….I’m here….What can I do for you?” The man looked over at Dave’s desk and exclaimed with an over emphasised amount of joy, “It looks to me like you’re trying to write a letter.” Dave nodded grimly, he hated himself for asking the idiot over but he really was stuck.
“So what seems to be the problem bub?” asked the Sorcerer.
Dave took hold of the letter opener on his desk and pointed it towards the ‘wizard’. “Call me bub again…..and….” he stumbled whilst he fought for the right words….”I’ll cut the point off that bloody hat of yours.” The Wizard looked stunned and slowly but delicately took off his black pointed hat and hid it behind his back. Dave sat back down at his desk and the wizard moved towards him…wheeling the chair slowly. “I’m just having trouble with one part,” said Dave. “I can’t for the life of me remember where the signature goes.”

The wizard suddenly flung him self back in his chair and roared with laughter. “Now that’s something I can help you with me-laddo,” he exclaimed. The letter opener was once again raised and wizard’s eyes widened slightly. After a short stab in the air by the angry office worker, the blunt blade was once again lowered. “Right, let’s have a look at what we have so far.” Dave reluctantly gave the sheet of paper to wizard. Wizard started looking over it, silently humming a happy little tune to himself. Dave started slowly and rhythmically banging his head on the desk. Wizard obviously couldn’t hear himself so he hummed a little louder, and Dave combined the head banging with the addition of fingers in his ears.

Wizard quickly took a glance around, made sure Dave wasn’t looking, and then gingerly pulled the front of his trousers away from his stomach. His other hand swiftly picked up Dave’s letter and stuffed it down his pants. He found a blank piece of paper and tapped Dave on the shoulder. “Shall we begin?” he said. Dave seemed a little shocked, but then nothing about the strange little man surprised him anymore.
“What do you mean, begin?” he asked. The little man gave a short strange little smile and then continued, “You’ve enlisted the help of a wizard now, we must start everything with a blank slate.” Dave looked angry. It wasn’t surprising, even though the letter wasn’t long it had taken him a fair amount of time to compose it.
“But…but…what happened to my letter?” He asked.
“Destroyed” said Wizard.
“WHAT!!” Shouted Dave. The little man was starting to really get on his nerves. The wizard tried to reassure him.
“Don’t worry we’ll start it all over again,” he chuckled to him self, “and get it right this time.” Dave was tired. It had taken all morning to write that letter and he wasn’t about to write it all over again by himself. “So, let’s see, first of all we need the letter body.”
Dave frowned “You mean the recipient?”
“No” Said Wizard. “We start with the message body”
Dave looked at him in disbelief. “What kinda screwed up way of writing a letter is that?” Unphased, the wizard just replied joyfully, “I don’t believe you’ve been to Wizard School……Nope…..Well I have”

The two of them worked for the next 15 minutes, Dave with his head on the desk, and Wizard writing feverishly. Dave managed a short glance at the paper. Then he suddenly stood up, pointed to the page and shouted, “What the heck is that?”
“Why that would be the letter ‘a’ good Sir,” replied Wizard.
“That’s a 9!!” spat Dave. Wizard tried to smudge it and start over the letter again.
“Oh man you’re making it worse!!! I swear if this takes much longer I’m going to give you such a beating.” shouted Dave.

When they’d finally finished the body of letter Wizard looked over at Dave and asked, “How do you want to sign off?”
Dave scratched his chin and replied, “With the warmest regards, sounds about right.”
Wizard’s face dropped. “Sorry”, he replied. “You can choose from either ‘With Love’, ‘Yours Sincerely’ or ‘Yours Faithfully’. Dave slammed his hands down on the desk. This was not going well. It had taken just as long to get the stupid wizard involved as it had for him to do half the letter himself.
“Why can’t I choose what I want to write” he asked in desperation.
Wizard patted him on the back as he stood there panting. “You’ve never written a letter using the wizard before have you?” Dave slumped in the chair and just replied “Yours Faithfully.”

“Your name?” asked Wizard.
“Don’t be stupid, dim wit!!” Was all Dave could reply.

“I’ll just put David,” said the magician. Dave sat up again and then slammed his hand down on the paper. He looked tired now. His eyes were wired and his hair a complete mess. The ironic thing is that the Wizard had supposed to have been a quick end to a long and boring job. Dave extended the blunt blade in threat once again.
Through gritted teeth David hissed, “I’ve written this letter twice now, thanks to you. I am signing it myself.”
Wizard began to protest, “I’m afraid Section 3.2 of the wizard code states that no user may input anything into the document itself until the Wizard has completed the task.” The head banging commenced once more accompanied this time by fists too.

“Now”, said Wizard, finishing off signing the letter from D-a-y-v-e-d. “Who is the delightful letter going to?”

Dave responded, “Jean Kiln, Michael Simmons, Marty Beanham….”
Wizard held up his hand. “Woah, Woah, Woah….” he laughed. “You said more than one name”
“Yes” replied Dave, once again dumbfounded at the weird little man. “That’s because I want to send it to more than one person”
“Oh, I’m afraid you can’t do that with a letter”, said Wizard.
“Why the hell NOT??” Dave’s forehead was throbbing now.
“Well”, started Wizard, “it’s not in the spirit of a letter. I think what you’re looking for is more like spam. That requires a level 3 wizard who’s studied in the ancient art of Advanced Correspondence”

Dave sat there for a few minutes. His brain wasn’t quite working. He couldn’t understand what this meant. What had he done to deserve this? “So what are my options?” He said, finally breaking the silence.
“Well you could start the letter again?” replied Wizard, getting up jostling his trousers.
“Can I have that piece of paper,” Dave started, “or is that a stupid question?” Wizard did the weird smile again, “Sorry”, he replied, “Official Wizard stationary.”
Wizard tried to subtly insert the sheet into his pants, but Dave noticed him this time. “You’re a ….. you’re a real weirdo!!!” he shouted. “What the heck are you doing now???”
The little man started wheeling away, but he replied none the less. “My personal shredder is broken, so under Wizard rules I have to put all data corresponding to your request somewhere where you will never be able to see them again.” He paused, stopped wheeling and then added, “I could have chosen to burn them I guess.”