Video Live Wallpaper, Part 4

I finally made some sample code available:

The code is uber-simple. It takes a video in the assets/video directory, copies it to the SD card, then draws it as fast as possible. Scrolling is not supported.

Performance depends on both the size of the video and the codec. (h263 seems to run faster than h264.) The test video is “Soft Boy”, a Blender movie clip that’s a DIVX 720×320 video. On my underpowered arm6 phone, I’m getting 20fps. Not bad.

Improving performance On the FFmpeg side, hardware acceleration seems to be a Google Summer of Code project. (I can’t find the link now.) In particular, when you load the video frame into pFrameConverted it’s currently done by the software. That’s one place that will benefit from hardware acceleration.

Right now, the video runs on two threads: the “main” thread and the rendering thread. However, the rendering thread actually does more than draw a video frame. It also gets the next frame (using FFmpeg). As per this post, we might see some performance improvement by splitting the rendering thread into two threads: one thread to get the next frame and another thread to draw the current frame.

Well that’s all for now. Hope you enjoy the code!

Part 1, Part 2, Part 3


71 thoughts on “Video Live Wallpaper, Part 4

    1. yes and no. to compile ffmpeg u have to use the commandline but thats straightforward. once you do that import the project into eclipse. make sure to manually add the lib directory containing glwallpaperservice.jar directory.

  1. By the way, amazing job explaining the overall logic on how to make this work. Very cool. I’m surprised more people aren’t saying thanks. Really good stuff that’s hard to find. Thanks dude.

  2. Thank you for your response. Much appreciated. My issue is when I go to compile ffmpeg using commandline. Everything compiles fine except at the end on the video.c file. I get a fail message but the other files compiled fine. I’ve tried several times in different ways, but same result, I get an error. Any ideas what may cause such a thing? Thanks.

    1. I just noticed that the compile instructions aren’t clear. So (1) compile FFmpeg using (Check, you got that part), and then (2) compile video.c using ndk-build. cd into the root project directory and then type “ndk-build”. You’ll get some warning errors but it should compile.

      In fact, I just re-downloaded, re-compiled, and re-installed the source from GitHub. It worked like a charm.

      Sorry for the late reply.

    1. So to be clear, the compiled version you made doesn’t work?

      Dunno what could be the cause, I just did a fresh re-install (see above comment) and it worked.

      Are you uninstalling the debug version?

      1. Sorry, I was a noob. I did not compiled files with NDK. So i got Exception Ljava/lang/UnsatisfiedLinkError; thrown while initializing Lfrankandrobot/glwallpapervideodemo/com/NativeCalls;

        But anyway, I have another problem, tried NDK 6rb and NDK r5b with Cygwin and Ubuntu 10 on VirtualBox. In all cases i got error, after running “ndk-build”

        Install : => libs/armeabi/
        /home/andrey/android-ndk-r5b/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-strip:/media/android_proj/frankandrobot-GLWallpaperVideoDemo-cad06ff/libs/armeabi/ File format not recognized
        make: *** [/media/android_proj/frankandrobot-GLWallpaperVideoDemo-cad06ff/libs/armeabi/] Ошибка 1

        What Iam doing wrong now? In project folder i can see now folder “libs” and “obj” with but it is not enough.

        Any advice?

  3. Also first of all i have error on first step: “compile FFmpeg using”

    I get:

    arm-linux-androideabi-gcc is unable to create an executable file.
    C compiler test failed.

  4. ikaruga2099, maybe you write step-by-step plan
    I downloaded ffmpeg, unpacked using, unpacking been warnings.
    Run failed. Uknown option “- soname-prefix =/…

  5. I have an issue where setting the wallpaper initially (works fine), change to another wallpaper, change back, the video does not start playing.

    Also video plays and shows in preview when setting paper, but when you click the “set Wallpaper” button from the preview, i just get a black screen, no video.

    Anyone else have this issue – or anything similar?

    1. yeah there’s a bug either in the jni code (not managing memory correctly) or in the renderer constructor. its on my Todo list

      1. when you reload a video, the old video doesn’t get deleted from memory correctly. i suspect its the way jni handles memory. was reading about that today. need to also double check that the opengl texture is deleted when the opengl context is active otherwise it doesn’t work and causes a crash

    2. Looks like “RendererDrawFrame(..)” is in logcat showing fps numbers and that the video is being processed, just not displayed.

      public void onDrawFrame(GL10 gl) {
      synchronized(lock) {
      //Log.d(TAG,”Drawing ….”);
      NativeCalls.getFrame(); // from video
      NativeCalls.drawFrame(); // using openGL
      if (MyDebug.showFPS) {
      final float fpsRate;
      fpsRate = 1000f/((float) (SystemClock.uptimeMillis() – fpsTime) );
      fpsTime = SystemClock.uptimeMillis();
      Log.d(TAG, “drawFrame(): fps: ” + String.valueOf(fpsRate) );

  6. mReportedVisible:true

    Notice this is showing in logs backward, not sure if that’s helpful to you or not.

    Meaning, when the wallpaper is visible, that returns mReportedVisible:false. Vice-versa, they point opposite.

  7. Sorry for the delay, doing some digging now, I had to go do some “real paying work” – such a nuissance! j/k

    Anyway, after doing my digging it just looks like the destruct method is not clearing your ‘runOnce’ flag.

    Simple testing (paste below) seems to fix it.

    void process(int width, int height) {
    setScreenDimensions( width, height );
    Log.d(TAG,”Killing texture”);
    setTextureDimensions( screenWidth, screenHeight );
    setFitToScreenDimensions( mParent.videoWidth, mParent.videoHeight );
    if ( true/*!runOnce*/ ) { // CHANGE HERE!
    Log.d(TAG,”VideoRenderer:process – Preparing frame”);
    runOnce = true;

    Just seems the storage frame was not getting set back up. (Like I said, quick checking here, may be more to it).

    Again, thank you for the code share!


    1. if i recall correctly fixing the destructor doesn’t work. but if you see the bug go away, i might be wrong. I’m in the same boat right now with paying work. been meaning to rework the code

  8. please give proper solution because In this code when i am compile jni so in command prompt i see error Format Not Recognized give me solution how to solve it

  9. Thanks for your fantastic blog.
    I really learned a lot from this. I am trying to run the project and getting some issues. The video in live wallpaper is loaded however it is not displayed immediately. Once I refresh the screen the video wallpaper appears. I am not sure where exactly I am going wrong.
    I think that once I load the wallpaper first time the pFrameConverted->data is not filled and hence draw frame could not display the same. Once the video loading is finished and then I refresh the screen video appears immediately.

    Can you please suggest where I am making mistake.

    Thanks a lot in advance!!

      1. Hi ,

        Videos are very small and low resolution. still the problem is appearing. Once video is loaded “loading live wallpaper…” disappears and blank screens is shown. anytime after that when I refresh the screen , video is displayed. Can you suggest something?

        thanks in advance!

  10. whoah this weblog is fantastic i really like reading your
    articles. Keep up the good work! You already know, a lot of people are looking round for this info, you can
    help them greatly.

  11. Thanks for these tutorials!

    I was suprised when I found a tutorial that was weirdly exactly what I was trying to do.
    And now I’m all up and running 🙂

  12. First, a very sincere thanks. I’ve spent a lot of hours searching for something like this, and a good 40+ hours trying to build something myself using individual frames, and was at wits end when someone sent me here.

    I’m having some trouble understanding how to get up and running. I learn best by taking working code and modifying it; it helps me separate each component and focus on each individually. However, after downloading this project from github and adding the jar file in the lib folder, I’m able to install it on my phone but receive an error when previewing. It’s a fatal exception on line 165: “NativeCalls.setSpanVideo(spanVideo);”.

    I didn’t do anything with the NDK or compiling FFMpeg. I read through the instructions in part 1, but they felt over my head and several comments implied that it couldn’t be done on Windows (and I have no access to a Unix box). I had assumed that the github project contained the already compiled libraries. Is this not the case, or is there something else I’m missing? I’m a relative newbie at both Java and Android (this is actually my first project), so it could be something boneheaded.

    Thanks again for making these posts, and thanks in advance for any response.

    1. Would you be available for assistance via email in getting this example up and running and answering a few questions? I’d be willing to pay even, though my budget is very limited. This is something I’d really like to be able to do, but the information I keep finding online is targeted at more experienced developers and I’m having trouble grasping the fundamentals. Thanks.

      1. I have managed to get this compiling correctly and am fighting with the screen spanning code right now ( I feel confident I can figure this out, but was hoping maybe I could get your insight on framerates.

        I’m using an Xperia Ion with a dual core 1.5 GHz process. However, even using the included softboy video I’m only getting some 14 fps. This seems very wrong given that your test on much older hardware came in at 50% higher than that. Would you happen to have any thoughts as to why this might be?

        Meanwhile, my Droid X (much older hardware) is reporting 20-22 fps, though it’s showing a black screen so I’m not sure how accurate that is.

      2. Thanks for responding.

        Forgive my ignorance, but I thought that the loading of the video frame was done in software, but the resizing to fit the screen was handled by GLWallpaperService which is hardware accelerated. Is this something that could easily be hardware accelerated using today’s technology? Could you point me in the right direction to research how to do this?

        Regarding my earlier comment, is updating this to support Android 2.3 – 4.x (and using hardware acceleration) something you’d be willing to do as contract work? If so, is it something easy enough for you that it would be reasonably priced?

      3. Correct. The loading which is actually decoding is done in software. The decoded frame is then rendered quickly with hardware. The.chokepoint is the decoding.

        Don’t think i have time for a side project right now but im willing to help u out if u do it open source.

        The best strategy is probably sofware decoding for old phones and renderscript for new phones.

      4. Thank you; I appreciate the offer of assistance. I’ll need to do some research to understand what renderscript is.

        I haven’t read too carefully through the license, but my understanding was that a requirement of using the code was that my modifications had to be open source. I’ve never done that before, but did plan to put it out there once my modifications are done.

        I’m a little confused about your explanation. You said that the decoding is software and rendering is hardware. However, isn’t the resizing to fit the screen done in the rendering stage? If so, why would my higher resolution phone slow down via decoding if the source frame is the same number of pixels regardless of target resolution? In other words, don’t both phones have the same number of pixels to decode, while the Ion just has more to render? And based on the full screen videos I watch online I’m assuming that full-fps rendering isn’t causing any sort of bottleneck.

        Thanks again!

      5. The performance issues appear to be tied to texture resolution. My Droid X defaults to 512×512 while the Ion tries for 1024. If I force the Ion to 512 the FPS jumps to almost 30, which is where it should be in comparison to the Droid.

        My assumption is that we’re first resizing the video to fit the texture in software and then resizing the texture to fit the screen in hardware. If that’s correct then I think we could see considerable performance gains by including two movie clips, one at 512px high and one at 1024px high, and then defaulting all devices to a 512px texture but with a setting to use the 1024 video with a 1024 texture. This allows us to remove the first scaling step. Does that make sense? If so, would you have any advice as to the required changes? I’m not sure I understand that section of the code.

        My Droid is experiencing the problem listed above where it appears to be playing back correctly and the logs report a framerate, but I get only a black screen. I tried the solution posted. Was there any further progress made on that issue? Was the bug ever found?

        One more question if you don’t mind. I noticed a line in the log that says “: Format RGB_565”. And it does look like the video is running in that smaller colorspace. Do you know where that setting can be changed? I can’t seem to figure it out.

        Thanks again!

      6. So basically you’re saying that the texture resolution is sometimes too large? So that means VideoRenderer#setTextureDimensions isn’t smart enough.

        We can test out your idea that the scaling is the cause by using a larger video on your Ion and seeing if its still slow.

        As for the RGA format issue, take a look at video.c. If I recall correctly, take a look at textureFormat and GL_RGBA. I seem to recall that I chose that format because it uses the least memory. Before playing with the format, make sure that your Driod supports GLWallpaperaService (which is what this is based on). Try one of these wallpapers first and see if everything is OK:!topic/glwallpaperservice/pDlj6iIuhxE

      7. I hadn’t considered a GLWallpaperaService incompatibility. Good suggestion. But I gave Biomechanical Skill Free a try and it worked just fine on the Droid X. Prior comments indicate it works the first time for them and then not again until phone restart. However, I tried that and it’s still not working for me.

        I compared the debug information between the two phones for a preview of the wallpaper. The Ion shows libraries loading before copying the video over while the Droid does not. The Ion also shows more D/libEGL libraries loading after surface creations, as well as memallocation and a “can’t open file for reading” error, all missing on the Droid. Could these missing messages indicate an initialization problem on the Droid?

        Regarding the texture sizing, I tried a couple other videos. One was twice the resolution of the softboy avi, but all of them ran with pretty similar framerates (about 10% difference). So I don’t think the video resolution is necessarily a limiting factor. Instead, I get the impression that it’s mapping the source (whatever resolution that happens to be) to each individual pixel of the texture that takes time. That’s why I suggested a video that matches the texture resolution; so no scaling is required. But I’m not sure which line does the scaling or how to replace it with a line that writes the frame directly to the texture without scaling.

        So I don’t think it’s a matter of VideoRenderer#setTextureDimensions not being smart enough. Rather it’s too smart. You designed it to work with any video, while I can tailor the video specifications for optimal quality. So I want to take away this procedure’s versatility and trade that for better performance.

        I did see textureFormat set to PIX_FMT_RGBA, but skipped over it because I thought PIX_FMT_RGBA mapped to full 32-bit color. The FFMpeg docs support that assumption: “PIX_FMT_RGBA packed RGBA 8:8:8:8, 32bpp, RGBARGBA…”. So there must be something somewhere else dropping us back to RGB_565. As an interesting side note, my Droid X does not show this line in the debug, so I’m thinking it may be an Xperia-only configuration thing.

        Also as an aside, I tried several other options listed in the FFMpeg docs for texture format (not really understanding the details). There are several that list hardware decoding, but all the ones I tried all show a blank black screen on my Ion. This is the exact problem my Droid has with the original code. So maybe it could have to do with what formats are supported on what devices (ie: my droid doesn’t support PIX_FMT_RGBA for some reason). Is this reasonable or am I way off?

        Thanks again.

      8. Thank you for all the help you gave me; I was able to get many of the bugs worked out and have a functioning video live wallpaper on my phone. However, I’m running into a problem changing the package name, which I will need to do for each new wallpaper. I have detailed it at StackOverflow at, but it essentially comes down to this error:

        145 could not load needed library ‘/data/data/’ for ‘’ (load_library[1091]: Library ‘/data/data/’ not found

        I’m guessing that there is more to changing a package name when using the SDK than when working with a standard Java app. I was hoping that you might be able to provide some insight.


      9. Thanks. Could you please tell me which C file? My understanding was that Eclipse would recompile those for me. I know I’ve made many changes to the video.c file over the past few weeks and they were always picked up automatically for me.

        Do the libraries need recompiled?

      10. Oh wait. Its ffmpeg that you need to recompile. It has the package name passed as a flag. Search for the name in the files inside the folder. Unfortunately, never tried compiling without that flag.

      11. Aha! Both Eclipse and I missed that on our search. Thanks. I’m writing up a set of instructions as we go so I can release them with the source code that I make available.

        I never compiled ffmpeg before as I simply downloaded your example code. In part one step 7 says “Compile FFmpeg. Run from the command line.” I’m on Windows 7 and downloaded Cygwin, learned how to browse out to my directory, and then tried running However, it just says “-bash: command not found”. I apologize if this is a naive question, but do I need a C compiler to make this work? Do you know of any good tutorials on this? I tried Google but kept getting stuff that was a little over my head.

      12. Yea. You need to install the buildenvironment for cygwin. And windows is notoriously ficky. Give me your package name. I’ll see if i can compile two versions. One with a package name and another without

      13. The package name is “com.nightscapecreations.orionkeysfree”. Thank you for taking the time to do that.

        I really hope the one without will work, as I plan to make a number of wallpapers with this code, and hope others can find a way to use it without having to recompile ffmpeg every time.

      14. I’m pretty sure the one w/o the package name will work.

        Status update: tried NDK 5b and 5c and was unable to compile the lib. It seems that the library in my project is missing some files.

        Searching for the right version of ffmpeg to redownload.

      15. Thanks for taking the time to compile them; I really apprecaite it. I noticed when trying to do the compile on my machine that I had a newer ndk and worried that might make a difference.

        I’m afraid I have to be the bearer of bad news. The packageless one did not work. Logcat reported:

        10-04 08:20:57.414: E/AndroidRuntime(19139): Caused by: java.lang.UnsatisfiedLinkError: Cannot load library: link_image[1936]: 145 could not load needed library ‘’ for ‘’ (load_library[1091]: Library ‘’ not found)

        However, when I tried the one with the package name my video popped right up. That’s a troubling development, as it means I’ll need to compile two new versions of the libraries for each new live wallpaper I want to make. Unless there’s some configuration that can be set somewhere to set a dynamic basepath derived from the package name.

        I’m going to try downloading the older version of the ndk if I can find it and see if I can get it to work on Windows. If I can’t, would you be open to doing some compiles for me from time to time (or a large batch at once)? I could pay a small fee for each one, saving me the 20+ hours I’d waste trying to get a linux machine setup, and letting you make some extra money from (what I assume to be) a few minutes of work here and there.

      16. No worries. You can buy me a coffee someday:) Give me the name of the packages now and I’ll upload them to dropbox. It literally takes me longer to upload them than compiling them

      17. Thanks; I sincerely appreciate it. Unfortunately, I don’t know the names yet as I just planned to create more animations as inspiration struck me (assuming my app doesn’t end up with 1 download/year or somesuch). Though my understanding of android is that the package name doesn’t really matter as long as it’s unique. Is that correct? If so, then based on the plan of providing both a free and a paid version of each wallpaper, I suppose it wouldn’t hurt to go with:


        If they sell well I hope to be able to do another every few weeks. It’s been years since I’ve seen more than a few views of my artwork per month. The idea of sharing it with thousands while having the opportunity to climb out of debt from the experience is incredibly satisfying. Though I’m realistic enough that I’d be happy with a few free pizzas. 😉

        Regarding the coffee, I’d be happy to. Though since I doubt I’ll ever see you in person do you happen to have a ‘donate’ link so one of your friends can buy you that coffee for me by proxy? If I am able to make a little money off these apps, it seems only fair that I should share it with the person who made that possible.

      18. Ok done. Added anim1, anim2, and threw in anim3.
        They should be done uploading to the original dropbox link in a few hours.
        Let me know if they don’t work.

      19. Thank you; I really appreciate it. They appear to be working beautifully (number 1 does, anyway). I’ll let you know when I publish the app in case you’d like to see it. 🙂

      20. I just wanted to post back, say thank you again, and let you know that the wallpaper is in the app store if you’d like to see it. The name is Orion Keys under the developer name Nightscape Creations. I’m working on a second one now. 🙂

        I finally found a way to force the texture to full 32-bit color, which eliminated the ugly banding issues I was having. I also found that my video exported from Premiere was horribly fuzzy and faded (shame for how much the program costs) and found a much better way to do it here: . The results from that process are unbelievably beautiful.

        I’ve never done anything with open source software so I’m not familiar with any of the tools out there. I’ve zipped up my project directory and will be placing it up for download from my website within the next few days. Forgive the messy code; it really was my first Android project. : /

    2. So I tried once again on the Droid and this time the libraries loaded; maybe last time LogCat wasn’t showing everything. However, the RBG_565 line, the memory allocation, and the “can’t open file” error are all still missing. I don’t think those are relevant, though (other than to support the colorspace issue as an Xperia-only problem)

  13. Hi IKARUGA2099,
    First of all congratulations for the great tutoria ! it’s an amazing piece of work!
    I am facing the same situation as NICHOLAS13: I do not have the necessary hardware to compile FFMPEG on Linux.
    I am sure you do not want this to become an habit 🙂
    but please could you compile them for me as well?
    The package name would be:
    I’ll buy you a beer, or better a very nice pizza if you happen to come over Milano.
    Thank you a lot for anything you can do for me!!!

      1. Woah! I’m really loving the te/laptemtheme of this site. It’s simple, yet effective. A lot of times it’s tough to get that “perfect balance” between usability and appearance. I must say you have done a excellent job with this. Also, the blog loads extremely quick for me on Internet explorer. Excellent Blog!

  14. Thank you IKARUGA2099.
    I have been able to put my hands on a Linux PC and compile by myself!!!
    Thank you very much again.
    And the offer for pizza and beer is always valid.

  15. Hi IKARUGA2099,
    I have been able to build and run successfully your project.
    Thanks, it works greatly!

    Please how is it possible to add the audio of the video???

    Thank you again for the great tutorial and I am waiting you for the promised pizza in Milan.


  16. Hi. I am getting this error, what ever I try, the class is just there, and it is also stated in the manifest. It is my 3rd day in Android development, and I can not solve this issue, I have imported the GLWallpaperVideoDemo-master project into eclipse, and added GLWallpaperService.jar to the Java built path library.It seems that it can not find the class GLWallpaperVideoDemo, however everything seems very normal. :/

    Error I get when I run it in a AVD, 720×1280 res, 32 heap, 1024Memory, 64 gb SD

    04-15 15:46:58.275: E/AndroidRuntime(2024): java.lang.RuntimeException: Unable to instantiate service java.lang.ClassNotFoundException: Didn’t find class “” on path: DexPathList[[zip file “/data/app/”],nativeLibraryDirectories=[/data/app-lib/, /system/lib]]

    thank you

  17. I know it’s been a long time, but I recently pulled this back out and am attempting to build a more comprehensive app based on it. Unfortunately, I can’t seem to get it to work with a new name in Android Studio and I was hoping you could help.

    I installed Android Studio, imported my old app (which used the anim1free library that you compiled for me), tried to run it on my phone and (with a bit of tweaking) everything worked great. The app compiled and ran as expected.

    I then did a search and replace to change the name to anim2free. Then I downloaded the libraries for anim2free that you still have on dropbox and copied the so files into jniLibs. I also copied all the files into jni/ffmpeg-android. Then I tried again. No luck. I even tried switching everything to the anim3free libraries thinking that the anim2free ones may not have been compiled right, but I’m still having the same issues. At this point my logCat reports:

    03-01 18:27:14.019 27751-27751/? E/AndroidRuntime﹕ FATAL EXCEPTION: main
    java.lang.UnsatisfiedLinkError: setSpanVideo
    at com.nightscapecreations.anim3free.NativeCalls.setSpanVideo(Native Method)
    at com.nightscapecreations.anim3free.VideoLiveWallpaper.onSharedPreferenceChanged(
    at com.nightscapecreations.anim3free.VideoLiveWallpaper.onCreate(

    I’m wandering if this has to do with video.c. I noticed that none of the zip files you provided included, but I also don’t ever remember compiling it back when I used Eclipse. I made all the appropriate changes to video.c this time around, but when I run my tests I never see a new appear in the jniLibs folder (even if I delete the old). So I’m wandering if the problem is that I’ve made all the right changes, but I’m still using an old version of that is pointing to anim1free names. Does it sound like that might lead to the above error? If so, how did video.c get compiled back when I originally built this in Eclipse a few years ago? I haven’t really done anything with Java/Android since then so I’m still as novice as ever.

    Thanks! I don’t want to impose; I’d be happy to pay you for your time. I just really want to get this new app created, and have been banging my head against this problem for over 10 hours…

    1. I just wanted to leave a note that I figured this out. Someone guided me in response to my SO question: At first I wasn’t compiling the video.c, then I was recompiling everything and getting duplicate-conflicts. I removed everything from jniLibs and let Android Studio do all the compiling and everything worked great.

      In case you are still reading this, would you be able to help me understand where hardware acceleration is occurring and where it isn’t? With a new 1280×720 video that’s only 3 MB I’m getting 10-15 FPS. But if I just open the MP4 file and play it on the device it runs at 30 FPS without blinking. So obviously something here is not fully optimized. I’d love to hear your insights before I start poking blindly around. 🙂

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s