NUI

The iOS is a great mobile OS. It has provided us developers with many accessible and standard technologies from the very first SDK. Alas, with the arrival of new hardware, we have begun to see the classical problem of device fragmentation and subtil changes in the SDK that breaks some features and existing, perfectly valid, pieces of code. The one I will tell you about today it a problem that started to appear in the OpenGL ES framework with OS 3.0 on iPhone 3GS.
There is a bug in the way the the iPhone 3GS handles displaying OpenGL backbuffer to the screen when we ask the device to keep the backbuffer valid after the copy to the display. This is very useful if you need to keep your rendering to a bare minimum by drawing only what has changed in between two frames. Of course, most 3D games and applications don't really care about this feature as the complete rendering changes in between two frames (just moving the camera a little obliges you to redraw everything). But when you render 2D graphics, as we do with NUI, it is a most needed optimization as most of the time, very little changes in between two frames. There are many algorithms in NUI to make sure that we send the real minimum draw operations to the graphic card and OpenGL. As long as you keep your rendering view in its original orientation, all is fine. But many applications want to adapt their view to the current orientation. In this case, the iOS does nothing for you: it will draw the contents of the back buffer on top of the new region by stretching it uglily. There is only one way to go back to a correct behaviour: recreate the back buffer, and that's were the crash happens: there is no way to recreate the back buffer of a copy on swap GL ES context without a crash on iOS 3.x / iPhone 3GS. All the other phones and OS combinations do work though. There are two options there: the first one is to have an anti bug in the code that disable device orientation on the configurations that are crashing, but this is ugly and we don't want to have to explain to the customers why oh why, their favourite app doesn't work with their device as expected. The second option is to find a solution that works every were. We chose the second option and we're glad to share it with you. You can check out the code here: http://redmine.libnui.net/projects/libnui/repository/show/trunk/nui3/scratchpads/testGLES_iOS and try it for your self.
The solution involves creating a dummy OpenGL ES view that is recreated whenever the orientation changes but in order to keep all the resources in memory we use the sharing group of the existing context to create the new one.

Please let us know if you have found a problem or a way to improve this solution! It has been adapted and implemented in the current NUI SVN head and reported to have passed the App Store's approval process so feel free to use it in your code.

Sebastien

Once more, BeatMaker, one of the flagship nui applications, has been featured by Apple on its website. This time it's for the iPhone 4 announcement!
Check It Out!

Congrats guys!

I have just pushed a big commit: all widgets and objects are effectively ref counted now.
If you want to keep a pointer on a widget/container/object in some of your class you must call pObject->Acquire(). When you are no longer interested in that object use pObject->Release(). This permits to reference widgets outside their parents without fearing them to disappear without being noticed about it.
The change is mostly automatic if you have no special code than would use DelChild(pWidget, false) or Clear(false) then you should be 100% safe. However I have changed DelChild and Clear: they don't give the option to not delete child widgets anymore. They will always release them. If you want to keep them alive you should change your code so that you Acquire then before hand.
I'll be checking for leaks and errors in this new implementation today so I would advise you to test your code against the current SVN head and let me know if you find anything strange. This code should be perfectly safe (i.e. 200% memory leak free) in a couple of days.

I will also try to add a smart pointer class that handles this reference counting automatically later today.

This change was needed to make the scripting language bindings much easier as most languages have garbage collectors that destroy object automatically when they are not referenced anymore.

We've been very busy fixing bugs and adding small features in many parts of the lib lately:
- many threading bugs have been fixed
- we have added some multi threading helpers like nuiReaderWriterLock that handle the multiple reader/one writer pattern and nuiCondition.
- many optimizations have been added to the whole lib and particularly concerning graphic rendering and the management of the widget tree. The general idea is to optimize for the iPhone without compromising on the set of offered features (if it is fast on the iPhone/iPad then it ought to be ultra fast on a desktop machine). For now have have been able to make almost no concession on that front which if very cool.

Last but not least, there have been a huge effort to open a new dimension of NUI programming by opening it to other languages than C++. We really want to add bindings for high level and dynamic languages (so called scripting languages).
There are many languages available, they are more or less easy to create bindings for and they all have their pros on cons. There is one family of these languages that even have many available implementations: JavaScript. It is also the most pervasive scripting language of all as it runs inside all web browsers so our choice has been to choose that language as the first one for which we will add bindings in NUI.

Doing bindings is a really tiresome operation and we'd rather not repeat the same boring work for every implementation of every language so we tried to have a more general solution to the problem. Thanks to the use of tamplates we have now a nice way to describe the internals of a C++ class, its methods and its types, in a very simple way. We have also a nice way to create an object and call its methods only thru a set of classes (nuiBindingManager, nuiClass, nuiFunction, etc.). This part has just been proven to work very well so we are now ready for the next step: binding the API description stored in these classes to the scripting language implementation.

There are three main families of opensource JavaScript implementations right now: JavaScripCore (KDE + Apple / Safari / iPhone), V8 (Google Chrome) and SpiderMonkey + TraceMonkey and it descendants (FireFox). The choice was not easy to do as V8 is really easy to bind to C++ but it comes without an interpreter which disqualifies it for the iPhone (JIT is not possible there). JavaScriptCore is very fast too and it embeds an interpreter but it is LGPL which would mean dragging DLL with every Windows NUI application, and moreover it is not a public API on the iPhone (it is only available thru Safari/WebKit but it doesn't expose its binding API on that platform). This leaves us with only one choice: SpideMonkey or one of its JIT fork: it is offered under a very liberal license which permits us to ship it in a statically linked binary, it has an interpreter and it is a very mature project. The only problem is that the Fast JIT implementation is not yet in the main trunk which means SpiderMonkey only use an interpreter and this makes it slow.

The plan is to bind to the slow spidermonkey first and to prot it to every platform currently supported by NUI. Then we will investigate the possibility to add the JIT implementation when possible (i.e. for Desktops). Then we will start investigating the possibility to add the glue for the other JavaScript engines as well as other languages (Lua, Python, Ruby and .NET/mono comes to mind) once a good part of the NUI API is exposed. We also plan to use the JavaScript bindings to add JacaScript capabilities to nuiHTMLView.

We welcome any help that cound help us make that whole process faster and safer by implementing the actual bindings for existing methods and then test it!

Feel free to contact us for any questions 9and if you\re willing to help.

We will hopefully update this page in a few days to report the initial success of this project!

Thank you very much for your attention,

Sébastien

We have just fixed the Linux build for 32bit and 64bit!
There is still some updating and fixing to do (the keyboard is half broken right now).
Is there any X11 wizard around to help us bring this port to the same level of excellence than the others?

David Wallin (bleep!BOX developer) has just released another iPhone app made with libNUI.
It's called bleep!Synth and we are very happy about it!

check it out: http://www.bleepsynthapp.com/

Just a short note to announce that we have updated the VSTPlugin tutorial so that it also builds and run on MacOSX.

The code is also a bit more clean and the UI looks a little bit better, nothing earth shattering thought ;-).

We have just added a new sample projet that shows how to implement an NUI User Interface in a VST Plugin.
This is a simple demonstrations of how to integrate NUI in a plugin. The main point is to create the main window as a child of the host provided plugin window which is exactly what this code does: http://redmine.libnui.net/projects/libnui/repository/entry/trunk/nui3/tutorials/VSTPlugin/src/sdeditor.cpp#L90
It also shows how to implement VST parameter handling with nuiAttributes.

The limitations:
- absolutely no eye candy for now :-).
- only tested with minihost.exe, the minimalistic host provided with the SDK.
- Win32 only for now.

The code is in the svn (in the nui3/tutorials/VSTPlugin directory)

Let us know what you think about it and, please, do file issues as you encounter them.

We have been very busy optimizing libnui lately and our effort has been very well rewarded as the current SVN head is much faster at rendering most applications than the previous version. The change is particularly important on MacOSX and iPhone but Windows and Linux should see some good improvements in frame rate.
We have also been fixing many small quirks on all platforms, polished the animation engine with new easing methods, multithreaded the HTML widget, added new CSS constructs to handle parameters during widget creation, etc.

Here are some of the things we are working on at the moment:
- more nuiHTMLView improvements.
- a video API
- Ogre3D integration
- scripting (V8 has been started, SpiderMonkey is investigated and hopefully once we have a basic construct more langages can be added easily!)
- more performance related fixes.
- VST/AU plugin sample code.
- more CSS niceties (the goal is direct events binding and animations in the CSS without typing one line of C++ code...)

If you haven't updated your repository in a while, now would a good time to do so. Don't forget to let us know what you think in the comments and in the forum!

We're working on adding a set of simple class the eases the integration of NUI in existing OpenGL toolkits. The first toolkit we are supporting is the pervasive GLUT!
We have created a basic class named nuiGLUTBridge that serves as a very simple hub to relay the information needed by NUI to handle the mouse and keyboard events, as well as drawing the widgets on screen.

You can have a look at how simple is now is to add NUI to a GLUT application:
GLUTBridge

This class is also a very good example for people who want to integrate NUI in other toolkit. We'll be happy to add support for SDL, Ogre3D, the Nebula Device, Unity3D or any other toolkit. Feel free to send patches and sample code to us!