README.charm4py

Building Charm++ shared library for Charm4py
============================================

- Build libcharm for non-SMP mode using 'charm4py' build target.
  For example, on a regular Linux machine:
  ./build charm4py netlrts-linux-x86_64 -j8 --with-production

  NOTE: We are currently requiring non-smp mode because in the most common Python implementation
  (CPython), multiple threads cannot run Python code concurrently due to the Global Interpreter Lock
  (GIL).

Adding modules (e.g. load balancers) to libcharm.so
---------------------------------------------------

Currently, modules need to be manually registered (there is a section in
init.C with #if CMK_CHARM4PY where this can be done), and the LIBCHARM_LIBS variable in the Makefile
needs to be modified to include the module.

Developer documentation
=======================

Chares that are defined and that run outside of the C/C++ runtime (e.g. Python objects in a Charm4py
program) require lightweight objects acting as counterpart inside the C/C++ runtime.
These C++ objects are:

  ReadOnlyExt, MainchareExt, GroupExt and ArrayElemExt

and their function is mainly to relay received messages to their external counterpart.

To allow use of the C++ runtime from external clients (e.g. Charm4py), changes to the
following files have been made:

charm++.h
  Declaration of ReadOnlyExt, GroupExt, MainchareExt

charm.h
  Declaration of several functions intended to be called from external client. This
  includes:
    - register callbacks to external client functions
    - register external Charm entities (Readonlies, Mainchare, Group, Array)
    - object creation
    - send message functions
    - hooks to Charm functions

register.C
  define registration functions for external Charm entities

ck.C
  implements most of the new functions and methods

ckarray.h/C
  ArrayElemExt

init.C
  Changes to circumvent linking issues (see 'Improve/support external module linking'
  below for details)
  NOTE: These changes are only enabled if building Charm with charm4py support.

TODO
====

Improve/support external module linking
---------------------------------------

There are modules like tracing and load balancers that the Charm++ runtime expects
to be linked at the time the final executable is generated. This includes
definitions of the following functions (at the least empty definitions of them):

void CkRegisterMainModule();
void _registerExternalModules(char **argv);
void _createTraces(char **argv);

These are typically defined in a moduleInit.C or similar file created by charmc
during linking of final program executable.

Problem is that when accesing the Charm shared library from Charm4py, these functions
are never defined, so I have disabled calls to them in some cases, or bypassed with alternative
code with #if CMK_CHARM4PY macro.

This means that I implemented an alternative way of registering the main
module, via a callback to external client (called from init.C).
And this also means that registering modules like load balancers has to be done
manually in the Charm code (e.g. in init.C).

This process should be improved.

A solution might be to define these functions in a Python c-extension module, and
maybe when accessing the shared library the functions are found, although this
method assumes that the external client will define them.

Or maybe modify charmc to allow building a shared library with the components that
the user wants predefined and linked in.