Monday, March 22, 2010

Share memory using ashmem and binder in the android framework

To share memory between different processes in the android framework ashmem can be used. Ashmem is a android shared memory addon to the linux kernel.
Ashmem has reference counting so that if many processes use the same area the area will not be removed until all processes has released it, the memory is also virtual and not physically contiguous. If physically contiguous memory is needed like for hardware reasons pmem can be used, but it has no reference counting and is not part of standard Android kernel.
In this blog post we will concentrate only on ashmem.

Ashmem is allocated via ashmem_create_region() that gives you an file descriptor then you use mmap() on it to get a memory area:

int fd = ashmem_create_region("SharedRegionName", size); if(fd == 0) {
  data = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
  if(data != MAP_FAILED)
  {
    /* It works do your stuff*/
  }
}

But this will not solve your problem on how to share the memory area. The first problem is that the memory pointer "data" in our example above is per process and can't be shared. The next problem is that for security reasons the name "SharedRegionName" in the example above are not shared between processes. The "other" process that want to access the same shared memory area can't use a ashmem_create_region() with the same name to get access to the same physical memory area. Instead the file descriptor (fd in the example above) needs to be used in a new mmap() in the "other" process. Here comes the third problem, the file descriptor are per process and can't be shared with the other process just like that. Bummer!

The solution is to share the file descriptor with the binder since the binder has special functions that can be used to transfer file descriptors over it's interface.
To help you with ashmem handeling and file descriptor sharing the class MemoryHeapBase can be used.

To transfer the MemoryHeapBase object you can use the asBinder() on the IMemoryHeap interface class on the server side to transfer the needed information to setup an new area on the client side mapped to the same physical memory.

sp<imemoryheap> memHeap = ...
reply->writeStrongBinder(memHeap->asBinder())

On the client side the MemoryHeapBase class will take care of all the needed stuff for you and all you need to do in your binder wrapper class is something like this:

sp<imemoryheap> memHeap;
memHeap = interface_cast<imemoryheap> (reply.readStrongBinder());


Please remember that the mmap:ed memories virtual address is not equal to the physical address even more important that the virtual address is different to each process and might be different for every call to the
getBufferMemPointer() function even if the same ashmem region is used so be careful if you have stored your pointer somewhere and still operate on it.

Here is a complete example setting up everything needed to get this working.

Interface header: IEneaBuffer.h
/*
 * IEneabuffer.h
 *  Created on: 19 mars 2010  Author: Zingo Andersen
 *  License: Public Domain (steal and use what you like)
 *
 * Buffer classes to handle the binder communication  */

#ifndef IENEABUFFER_H_
#define IENEABUFFER_H_

#include <utils/RefBase.h>
#include <binder/IInterface.h>
#include <binder/Parcel.h>
#include <binder/IMemory.h>
#include <utils/Timers.h>

namespace android {

class IEneaBuffer: public IInterface
{
public:
  DECLARE_META_INTERFACE(EneaBuffer);
  virtual sp<IMemoryHeap>           getBuffer() = 0;
};

/* --- Server side --- */

class BnEneaBuffer: public BnInterface<IEneaBuffer>
{
public:
  virtual status_t    onTransact( uint32_t code,
                                  const Parcel& data,
                                  Parcel* reply,
                                  uint32_t flags = 0);
};

}; // namespace android

#endif /* IENEABUFFER_H_ */

Interface class: IEneaBuffer.cpp
/*
 * IEneaBuffer.cpp
 *  Created on: 19 mars 2010  Author: Zingo Andersen
 *  License: Public Domain (steal and use what you like)
 *
 * Buffer classes to handle the binder communication
 */

//#define LOG_TAG "IEneaBuffer"
#include <utils/Log.h>
#include <stdint.h>
#include <sys/types.h>
#include <binder/MemoryHeapBase.h>
#include <IEneaBuffer.h>

namespace android {

enum {
    GET_BUFFER = IBinder::FIRST_CALL_TRANSACTION
};

/* --- Client side --- */
class BpEneaBuffer: public BpInterface<IEneaBuffer>
{
public:
  BpEneaBuffer(const sp<IBinder>& impl) : BpInterface<IEneaBuffer>(impl)
    {
    }

  sp<IMemoryHeap> getBuffer()
  {
    Parcel data, reply;
    sp<IMemoryHeap> memHeap = NULL;
    data.writeInterfaceToken(IEneaBuffer::getInterfaceDescriptor());
    // This will result in a call to the onTransact()
    // method on the server in it's context (from it's binder threads)
    remote()->transact(GET_BUFFER, data, &reply);
    memHeap = interface_cast<IMemoryHeap> (reply.readStrongBinder());
    return memHeap;
  }
};

IMPLEMENT_META_INTERFACE(EneaBuffer, "android.vendor.IEneaBuffer");

/* --- Server side --- */

status_t BnEneaBuffer::onTransact(uint32_t code, const Parcel& data, Parcel* reply, uint32_t flags)
{
  switch (code)
  {
    case GET_BUFFER:
    {
      CHECK_INTERFACE(IEneaBuffer, data, reply);
      sp<IMemoryHeap> Data = getBuffer();
      if (Data != NULL)
      {
        reply->writeStrongBinder(Data->asBinder());
      }
      return NO_ERROR;
      break;
    }
    default:
      return BBinder::onTransact(code, data, reply, flags);
  }
}

}; // namespace android

Then use it in your server class by inherit the server class with something like this:

Server command: EneaBufferServer.cpp
/*
 * EneaBufferServer.cpp
 *  Created on: 19 mars 2010  Author: Zingo Andersen
 *  License: Public Domain (steal and use what you like)
 *
 *  The Server will create a shared area that the client will then use
 *  The server will initiate the first int in the area and print it's value every
 *  5s. If you start the client in parallell it will try to change this value
 *  (value=value+1).
 */


#include "IEneaBuffer.h"
#include <binder/MemoryHeapBase.h>
#include <binder/IServiceManager.h>
#include <binder/IPCThreadState.h>

namespace android {

#define MEMORY_SIZE 10*1024  /* 10Kb shared memory*/

class EneaBufferService : public BnEneaBuffer {
public:
  static void instantiate();
  EneaBufferService();
  virtual ~EneaBufferService();
  virtual sp<IMemoryHeap> getBuffer();
private:
  sp<MemoryHeapBase> mMemHeap;
};

sp<IMemoryHeap> EneaBufferService::getBuffer()
{
  return mMemHeap;
}

void EneaBufferService::instantiate()
{
  status_t status;
  status = defaultServiceManager()->addService(String16("vendor.enea.Buffer"), new EneaBufferService());
}

EneaBufferService::EneaBufferService()
{
  //The memory is allocated using a MemoryHeapBase, and thereby is using ashmem
  mMemHeap = new MemoryHeapBase(MEMORY_SIZE);
  unsigned int *base = (unsigned int *) mMemHeap->getBase();
  *base=0xdeadcafe; //Initiate first value in buffer
}

EneaBufferService::~EneaBufferService()
{
  mMemHeap = 0;
}


static sp<IMemoryHeap> receiverMemBase;

unsigned int * getBufferMemPointer(void)
{
  static sp<IEneaBuffer> eneaBuffer;

  /* Get the buffer service */
  if (eneaBuffer == NULL)
  {
    sp<IServiceManager> sm = defaultServiceManager();
    sp<IBinder> binder;
    binder = sm->getService(String16("vendor.enea.Buffer"));
    if (binder != 0)
    {
      eneaBuffer = IEneaBuffer::asInterface(binder);
    }
  }
  if (eneaBuffer == NULL)
  {
    LOGE("The buffer service is not published");
    return (unsigned int *)-1; /* return an errorcode... */
  }
  else
  {
    receiverMemBase = eneaBuffer->getBuffer();
    return (unsigned int *) receiverMemBase->getBase();
  }
}


}

using namespace android;

int main(int argc, char** argv)
{
 unsigned int *base;

    EneaBufferService::instantiate();

    //Create binder threads for this "server"
    ProcessState::self()->startThreadPool();
    LOGD("Server is up and running");

    base = getBufferMemPointer();


    for(;;)
    {
     LOGD("EneaBufferServer base=%p Data=0x%x", base,*base);
     sleep(5);
    }
    // wait for threads to stop
    //   IPCThreadState::self()->joinThreadPool();
    return 0;
}



You need add the following line to register the service in the file frameworks/base/cmds/servicemanager/service_manager.c

{ AID_MEDIA, "vendor.enea.Buffer" },

On the client process side you can use something like the code snippet below to get the memory pointer.
NOTE: MemoryHeapBase is based on strong pointer and will be "magical" ref counted and removed thats why the client put the object pointer in a static variable in the example below to keep it from being removed.

/*
 * EneaBufferClient.cpp
 *  Created on: 19 mars 2010  Author: Zingo Andersen
 *  License: Public Domain (steal and use what you like)
 *
 *  Get the shared memory buffer from the server and change the first int value
 *  by adding one to it. The Server should be running in parallell pleas view
 *  the logcat for the result
 */


#include "IEneaBuffer.h"
#include <binder/MemoryHeapBase.h>
#include <binder/IServiceManager.h>

namespace android {
static sp<IMemoryHeap> receiverMemBase;

unsigned int * getBufferMemPointer(void)
{
  static sp<IEneaBuffer> eneaBuffer = 0;

  /* Get the buffer service */
  if (eneaBuffer == NULL)
  {
    sp<IServiceManager> sm = defaultServiceManager();
    sp<IBinder> binder;
    binder = sm->getService(String16("vendor.enea.Buffer"));
    if (binder != 0)
    {
      eneaBuffer = IEneaBuffer::asInterface(binder);
    }
  }
  if (eneaBuffer == NULL)
  {
    LOGE("The EneaBufferServer is not published");
    return (unsigned int *)-1; /* return an errorcode... */
  }
  else
  {
    receiverMemBase = eneaBuffer->getBuffer();
    return (unsigned int *) receiverMemBase->getBase();
  }
}

}

using namespace android;

int main(int argc, char** argv)
{
 // base could be on same address as Servers base but this
 // is purely by luck do NEVER rely on this. Linux memory
 // management may put it wherever it likes.
    unsigned int *base = getBufferMemPointer();
    if(base != (unsigned int *)-1)
    {
     LOGD("EneaBufferClient base=%p Data=0x%x\n",base, *base);
       *base = (*base)+1;
       LOGD("EneaBufferClient base=%p Data=0x%x CHANGED\n",base, *base);
        receiverMemBase = 0;
    }
    else
    {
     LOGE("Error shared memory not available\n");
    }
    return 0;
}



And my Android.mk file for this:
# Ashmem shared buffer example
# Created on: 19 mars 2010  Author: Zingo Andersen # License: Public Domain (steal and use what you like)

LOCAL_PATH:= $(call my-dir)

#
# BufferServer
#

include $(CLEAR_VARS)

LOCAL_SRC_FILES:=        \
 IEneaBuffer.cpp      \
 EneaBufferServer.cpp \
   
LOCAL_SHARED_LIBRARIES:= libcutils libutils libbinder

LOCAL_MODULE:= EneaBufferServer

LOCAL_CFLAGS+=-DLOG_TAG=\"EneaBufferServer\"

LOCAL_PRELINK_MODULE:=false
include $(BUILD_EXECUTABLE)

#
# BufferClient
#

include $(CLEAR_VARS)

LOCAL_SRC_FILES:=        \
 IEneaBuffer.cpp      \
 EneaBufferClient.cpp \
   
LOCAL_SHARED_LIBRARIES:= libcutils libutils libbinder

LOCAL_MODULE:= EneaBufferClient

LOCAL_CFLAGS+=-DLOG_TAG=\"EneaBufferClient\"

LOCAL_PRELINK_MODULE:=false
include $(BUILD_EXECUTABLE)


/Rebecka & Zingo

7 comments:

  1. Hi Rebecka & Zingo,
    it seems that whenever you publish code snippets including the template sp, the type-name is mistakenly interpreted as a non-existing HTML-tag. I.e. the type-name will not be shown.

    You need to use & g t ; and & l t ; (without the spaces in-between) instead of > and < when publishing.

    Besides from that, thanks for your excellent work!

    /Michael

    ReplyDelete
  2. Fixed, thanx for the input, missed that the <pre> tag didn't take care of this, sorry.

    ReplyDelete
  3. Hi members,
    I am new android technology, can any one provide me detailed information regarding (ashmem, binder, alarm, power manager). what is the purpose of these driver in android kernel.

    Regards
    Kolanchinathan S
    kolanchi19@gmail.com

    ReplyDelete
  4. Hi, Thank you for the example code. I was able to get this compiled for android 2.3 but I am not sure what to do about registering the service with the service manager:
    { AID_MEDIA, "vendor.enea.Buffer" },



    Will I need to recompile all the android source to make this change?


    When I try to start the server (without adding the service to service_manager) the status code returned in the instantiate method is -1.
    status = defaultServiceManager()->addService(String16("vendor.enea.Buffer"), new EneaBufferService());

    Thank you.

    ReplyDelete
  5. Hi, Rebecka & Zingo.
    It compliments for the optimal supplied guide. Taking cue from your example, I must develop an example that functions with Android. The problem is that I do not know like making and from where beginning. I have felt to speak about NDK and JNI but I do not know like creating the interface in Java.
    If give a hand to me, of I would be happy.
    Thanks
    PAX

    ReplyDelete
  6. an example of hooking this up with java would be nice, unless there's an easier way to use ASHMEM in java that I haven't been able to find yet. would be really handy for transfering large data blobs e.g. graphics. Also I'm looking at doing an xorg implemntation for gnu/linux on rooted devices that uses something like ASHMEM/IPC to transfer the virtual back buffer into something which is android gui capable, instead of the current nasty VNC/RDP methods which are slow and cumbersome (all those memory coppies, yuk!)

    ReplyDelete
  7. Thanks, I m going to try this with the python interpreter for android, it looks like indeed I do need to build the service into the rom. Thats fine, basically I want to replace the use of socket rpc server in the sl4a facade server with an ashmem ipc implementation, I would like the service to share to the interpreters, this way the interpreters/shells (sh, bean shell, lua, python, jruby etc...) can use the service providing the Android api via ashmem to the interpreters. One thing I am wrestling with though is the Camera facade and other media api's, I would like python to be able to perform operations/calculations on a stream of video using the opencv modules, but that would mean that python would actually need those Java arrays ( I think it's a Java arrays ) that the opencv and pil modules use to allow python to perform operations on them. I think this would be the best bet, case ipc is request for video stream, allocate enough memory for it and return/send the pointer to the memory location ( I think this is how it works from a high level overview) to python so it will have access to the resource.

    ReplyDelete