Skip to content

User Guide 0.3

Jump to bottom Edit New page
szegedi edited this page Jul 21, 2012 · 5 revisions

API documentation

Comprehensive API documentation in Javadoc format is available.

What's new

If you used the version 0.2 of the library, see What's new in 0.3. Using the library

There are two use cases for this library. First use case is when you want to dynamically link your code to other code. You might be writing a scripting shell or some other similar piece of work. The library provides you with a very low-barrier entry point, in the form of the class org.dynalang.dynalink.DefaultBootstrapper and its publicBootstrap method. Let's suppose that you use the ASM 4 library to generate the JVM bytecode of your classes, and you want to emit an invokedynamic call for a property getter "color" that you'd expect to return a string. Here's what you do:

mv.visitIndyMethodInsn("dyn:getProp:color", "(Ljava/lang/Object;)Ljava/lang/String;",
    new MethodHandle(MethodHandle.REF_invokeStatic, "org/dynalang/dynalink/DefaultBootstrapper",
    "publicBootstrap", MethodType.methodType(CallSite.class,
        MethodHandles.Lookup.class, String.class, MethodType.class, Object[].class).toMethodDescriptorString()),
    new Object[0]);

Of course, if you do emit more than one dynamic call site (i.e. want to retrieve both "color" and "shape" properties dynamically), you might want to refactor the object as:

import org.objectweb.asm.MethodHandle;
import static org.objectweb.asm.Opcodes.MH_INVOKESTATIC;
...
private static final MethodHandle BOOTSTRAP = new MethodHandle(MH_INVOKESTATIC, 
    "org/dynalang/dynalink/DefaultBootstrapper","publicBootstrap", MethodType.methodType(CallSite.class,
        MethodHandles.Lookup.class, String.class, MethodType.class, Object[].class).toMethodDescriptorString());
private static final Object[] BOOTSTRAP_ARGS = new Object[0];
...
mv.visitIndyMethodInsn("dyn:getProp:color", "(Ljava/lang/Object;)Ljava/lang/String;", BOOTSTRAP, BOOTSTRAP_ARGS);
mv.visitIndyMethodInsn("dyn:getProp:shape", "(Ljava/lang/Object;)Ljava/lang/String;", BOOTSTRAP, BOOTSTRAP_ARGS);

That's all there is to it! The system will do all the heavy lifting associated with finding and linking the exact code for the property getters based on the type of the argument passed in. Subsequent invocations with the same type will be fast as they'll go to the already linked method, and if the call site encounters a different type of an argument, it will silently repeat the linking process for the new type for you. Your code should equally query the color and the shape of a Ruby, JavaScript, or Python object, or for that matter, any Plain Old Java Object that happens to have a getColor() and getShape() methods.

If you look at the JavaDoc for the DefaultBootstrapper class, you'll see that it exposes two methods: bootstrap and publicBootstrap. The difference between the two is that bootstrap passes down the access context (the MethodHandles.Lookup object) it receives from the call site to the linkers, while publicBootstrap ignores it and always uses MethodHandles.publicLookup() instead. If your language runtime is happy with only accessing public methods, fields, and properties on POJOs, publicBootstrap gives you somewhat more lightweight call sites. Note that in the 0.3 version of the library, we don't even expose non-public members in the POJO linker; we do plan on doing it in the future, and there's no reason an existing language runtime couldn't already do it, so we already provide for the distinction on the bootstrap level.

If you're writing a language runtime

If you're writing a language runtime that exposes its own special kinds of objects, you will still want to generate all your call sites as above. However, in addition to being able to link dynamically to code in any language, you will also want the ability for any call sites to link to your own code. In simple terms, you want other users of Dynalink to be able to query your objects for their color and shape...

To achieve that, you need to go to the org.dynalang.dynalink.linker package, and specifically you need to implement the GuardingDynamicLinker interface. Additionally, when you package your language runtime in a JAR file, you will need to add the fully qualified class name of your implementation to the file named META-INF/services/org.dynalang.dynalink.linker.GuardingDynamicLinker file, as Dynalink uses the JAR service mechanism to discover and automatically load all language-specific linkers in the classpath.

You can keep using the DefaultBootstrapper, as Dynalink will find your own linker and load it if it is declared in the JAR file, and link your code with it. However, when you are creating a linker for your own use, you might want to avoid it and explicitly manage your own DynamicLinker instance. DynamicLinker is an object that ties together all loaded GuardingDynamicLinker implementations in the JVM, and is used internally by the DefaultBootstrapper to perform linking. You can, however, create your own customized instance too, and make sure to configure it so that your own language linker has the highest priority (is tried first when linking). That can give your code a performance edge. To do that, you will need to have code like this somewhere in your language runtime to provide the DefaultBootstrapper replacement functionality:

import org.dynalang.dynalink.*;
import org.dynalang.dynalink.linker.*;

public class AwesomeBootstrapper {
    private static final DynamicLinker dynamicLinker;
    static {
        final DynamicLinkerFactory factory = new DynamicLinkerFactory();
        final GuardingDynamicLinker awesomeLinker = new AwesomeLinker();
        factory.setPrioritizedLinker(awesomeLinker);
        dynamicLinker = factory.createLinker();
    }

    public static CallSite bootstrap(MethodHandles.Lookup caller, String name, MethodType type) {
        return dynamicLinker.link(
            new MonomorphicCallSite(CallSiteDescriptorFactory.create(caller, name, type));
    }

The factory is smart enough that even if it discovers the AwesomeLinker class through the JAR service mechanism, it will ignore it if you supplied a pre-created prioritized instance. Now all you have to do is use your org/awesomelang/AwesomeBootstrapper class name instead of org/dynalang/dynalink/DefaultBootstrapper class name when specifying bootstrap method names in call sites (i.e. in the above ASM 4 example).

Guarding linker?

Yes, the interface is named GuardingDynamicLinker. It has a sole method with this signature:

public GuardedInvocation getGuardedInvocation(LinkRequest linkRequest,
    LinkerServices linkerServices);

It is invoked for a particular invocation at particular call site. It needs to inspect both the call site (mostly for its method name and types) and the actual arguments and figure out whether it can produce a MethodHandle as the target for the call site. The call site descriptor and the arguments are passed in the LinkRequest object. In ordinary circumstances, you'll check something along the lines of:

if(linkRequest.getReceiver() instanceof AwesomeObject)

If not, return null -- the master linker will then ask the next (if any) guarding linker. This is the base requirement for cross-language interoperability; you only deal with what you know, and pass on what you don't. On the other hand, if you know what to do with the receiver object, then you'll produce a method handle for handling the call and a guard method handle.

Actually, the GuardedInvocation class above is nothing more than a value class, a triple of two method handles (one for the invocation, one for the guard condition) and a java.lang.invoke.SwitchPoint. Since your method handle is only valid under certain conditions (i.e. linkRequest.getReceiver() instanceof AwesomeObject), you will want to create a guard expressing this condition. The master linker will pass the guard and the invocation to the call site, which will compose them into a new method handle according to its inline caching strategy. I.e. the MonomorphicCallSite will create a guardWithTest() of the guard and the invocation, with fallback to the master linker's relink() method when the guard fails or switch point is invalidated. The main takeaway is that you needn't deal with any of that; just need to provide the invocation and the guard and/or a switch point.

You can use the switch point in your linker implementation if you want the ability to invalidate the guarded invocations asynchronously when some external condition changes. You just need to pass the switch point in your guarded invocation, and in the chosen event, invalidate it. You don't need to worry about invoking SwitchPoint.guardWithTest(); it is the job of the call site implementation to compose your invocation, the guard, and the switch point into a composite method handle that behaves according to the call site semantics (i.e. the MonomorphicCallSite class will relink itself on next invocation after you invalidate the currently linked method's switch point).

As for the MonomorphicCallSite: it's a call site that has a monomorphic inline cache, meaning it'll always just keep the last linked method until it's invalidated, and then relink. It's the simplest linking strategy possible, but it's not always the fastest in the long run. The library also comes with another implementation, ChainedCallSite which maintains a cascading list of several method handles to try, and usually results in better performance, but YMMV.

Call site descriptors

The library uses the concept of a "call site descriptor" throughout. All call site implementations have them, and the linkers will expect them. A call site descriptor is an immutable value representing the name of the method invoked, the signature at the call site, and the lookup that was used. It's basically a value tying together all the values passed to a bootstrap method. CallSiteDescriptor is an interface, and there is a CallSiteDescriptorFactory that creates default implementations. It canonicalizes instances, so all call sites that express the "get the property color using public lookup with signature Object(Object)" operation will receive the same descriptor. It is recommended that you use this factory, unless the call sites in the bytecode emitted by your language runtime have additional bootstrap arguments in them. If they do, you'll need to create your own call site descriptor implementations to store these additional arguments in them.

Advanced topic: type-based linkers

You can declare that your linker is the authoritative linker for all objects of a certain type. To do that, you need to implement the TypeBasedGuardingDynamicLinker interface that adds another method to the GuardingDynamicLinker interface:

public class AwesomeLinker implements TypeBasedGuardingDynamicLinker {
    public boolean canLinkType(Class<?> type) {
        return AwesomeObject.class.isAssignableFrom(type);
    }

    public GuardedInvocation getGuardedInvocation(LinkRequest linkRequest, LinkerService linkerServices) {
        final Object object = linkRequest.getArguments()[0];
        if(!(object instanceof AwesomeObject)) {
            return null;
        }
        AwesomeObject target = (AwesomeObject)object;
        ...
    }
}

Note that you still need to check whether you received a correct kind of object in your getGuardedInvocation method. The canLinkType can be used by the bacground framework as a dispatch optimization, but it is not guaranteed it will be.

What's LinkerServices?

It's an interface provided to your linker with some extra methods your linker might need. Currently it provides you with a asType() method that looks much like MethodHandle.asType(), except it will also inject language specific implicit type conversions when they are available in addition to the JVM specific ones. It also provides a getTypeConverter() method -- more about that too in a bit.

Cool, I want to define my own language type conversions!

Sure thing. Just have your GuardingDynamicLinker also implement the optional GuardingTypeConverterFactory interface. The linker framework will pick it up and do the rest of its magic to make sure it ends up in the call path when needed, as optimally as possible.

public class AwesomeLinker implements TypeBasedGuardingDynamicLinker, GuardingTypeConverterFactory {
    private static final MethodHandle CONVERT_AWESOME_TO_BOOLEAN = ...;
    private static final GuardedInvocation GUARDED_CONVERT_AWESOME_TO_BOOLEAN = 
        new GuardedInvocation(
            CONVERT_AWESOME_TO_BOOLEAN, 
            Guards.isInstance(AwesomeObject.class, CONVERT_AWESOME_TO_BOOLEAN.type());
    public GuardedInvocation convertToType(Class<?> sourceType, Class<?> targetType) {
        if(AwesomeObject.class.isAssignableFrom(sourceType) && Boolean.class == targetType) {
            return GUARDED_CONVERT_AWESOME_TO_BOOLEAN;
        }
        return null;
    }
}

Also, if you define your own type conversions, you'll almost certainly want to also implement the ConversionComparator interface. While Dynalink is great at finding the best applicable method among overloaded methods when linking to POJOs, if you provide your own language-specific conversions, then it sometimes might be able to apply your language's values to several types that are unrelated in Java. In that case, you need to help it by resolving the relative priority of possible type conversions. As a practical example, consider JavaScript. Say that you can pass a function in place of a single-method interface. Let's assume we're trying to pass a Runnable to a Thread constructor:

new Thread(function() {
  print("Look Ma, I'm running asynchronously!")
}).start()

If your JavaScript runtime provides conversion from any value to String (as it should, all JavaScript objects are convertible to strings) and also from any function to any single-method interface, the poor POJO linker will get confused because now the above function object can be applied to both new Thread(String name) and new Thread(Runnable target). You'll need to have your linker also implement ConversionComparator to resolve this conflict, obviously telling the POJO linker to favor functions being converted into single-method interfaces instead of to strings.

Finally, the Metaobject Protocol

Finally, what kind of invocations to provide? What method names and signatures to expect and react to? Also, what method names and signatures to emit in your own invokedynamic instructions? For purposes of interoperability, we'll reserve the method namespace dyn for the commonly-understood MOP, meaning every method name will start with dyn:. Also note that when we use the INVOKEDYNAMIC instruction, for sake of brevity we omit the business of specifying a bootstrap method that we already explained how to do previously.

The operations are:

  1. Get property of an object with a constant name

    Template: "dyn:getProp:${name}"(any-object-type)any-type

    Example:

    • Source code:

        obj.temperature

    • Bytecode:

        ALOAD 2 # assume obj is in 2nd local variable
  INVOKEDYNAMIC "dyn:getProp:temperature"(Ljava/lang/Object;)Ljava/lang/Number;

    Your GuardingDynamicLinker should recognize dyn:getprop:name as a property getter for a fixed name. MethodHandles.convertArguments() or even MethodHandles.filterArguments() for custom value conversions might of course be necessary both for receiver and return value.

  2. Set property of an object with a constant name

    Template: "dyn:setProp:${name}"(any-object-type,any-type)V

    Example:

    • Source code:

        obj.temperature = 1;

    • Bytecode:

        ALOAD 2
  ICONST_1
  INVOKEDYNAMIC "dyn:setProp:temperature"(Ljava/lang/Object;I)V;

    Your GuardingDynamicLinker should recognize dyn:setprop:name as a property setter for a fixed name. MethodHandles.convertArguments() or even MethodHandles.filterArguments() for custom value conversions might of course be necessary both for receiver and return value.

  3. Get property of an object with a non-constant identifier

    Template: "dyn:getProp"(any-object-type,any-type)any-type

    Example:

    • Source code:

        var a = "temperature"; obj[a]

    • Bytecode:

        ALOAD 2 # assume 'obj' is in 2nd slot
  ALOAD 3 # assume 'a' is in 3rd slot
  INVOKEDYNAMIC "dyn:getProp"(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Number;

    Your GuardingDynamicLinker should recognize dyn:getprop as a property getter for a name that can change between invocations, and which is passed in the arguments to the method handle. You probably shouldn't return a method handle that is fixed for the current value of the identifier (albeit you could if you also build the assumption into the guard). The expectation is that this will result in too frequent relinking, so you'd rather return a method handle that uses the value of the name. MethodHandle.asType() or even MethodHandles.filterArguments() for custom value conversions might of course be necessary. Note how the identifier argument can be of any type and is not restricted to a java.lang.String. The reasoning behind this is that not every language can prove the value will be a string at invocation time, and the language semantics can actually allow for, say, numeric IDs. Consider this in JavaScript:

     function x(d) {
     var arrayAndDict = ["arrayElement"];
     arrayAndDict.customProperty = "namedProperty";
     return arrayAndDict[d ? 0 : "customProperty"];
 }

    x(true) returns "arrayElement", x(false) returns "namedProperty". At the point of invocation, the type of the property identifier is not known in advance.

  4. Set property of an object with a non-constant identifier

    Template: "dyn:setProp"(any-object-type,any-type,any-type)V

    Example:

    • Source code:

        var a = "temperature"; obj[a] = 1

    • Bytecode:

        ALOAD 2 # assume 'obj' is in 2nd slot
  ALOAD 3 # assume 'a' is in 3rd slot
  ICONST_1
  INVOKEDYNAMIC "dyn:setProp"(Ljava/lang/Object;Ljava/lang/Object;I)V

    Your GuardingDynamicLinker should recognize dyn:setprop as a property setter for a name that can change between invocations. MethodHandle.asType() or even MethodHandles.filterArguments() for custom value conversions might of course be necessary. Concerns about binding the method handle to the identifier expressed in point 3 fully apply, as well as the reasoning behind allowing any type for the identifier.

  5. Get element of a container object

    Template: "dyn:getElem"(any-object-type,any-type)any-type

    Example:

    • Source code:

        var a = "temperature"; obj[a]

    • Bytecode:

         ALOAD 2 # assume 'obj' is in 2nd slot
   ALOAD 3 # assume 'a' is in 3rd slot
   INVOKEDYNAMIC "dyn:getElem"(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Number;

    Very similar to 3, except it can be used by languages that distinguish between namespaces of properties and keyspaces of container objects (arrays, lists, maps). All considerations in 3 apply. Additionally, if your language makes no distinction between the two, your GuardingDynamicLinker should respond to dyn:getElem identically as it would to dyn:getProp.

  6. Set element of a container object

    Template: "dyn:setElem"(any-object-type,any-type,any-type)V

    Example:

    • Source code:

        var a = "temperature"; obj[a] = 1

    • Bytecode:

        ALOAD 2 # assume 'obj' is in 2nd slot
  ALOAD 3 # assume 'a' is in 3rd slot
  ICONST_1
  INVOKEDYNAMIC "dyn:setElem"(Ljava/lang/Object;Ljava/lang/Object;I)V

    Very similar to 4, except it can be used by languages that distinguish between namespaces of properties and keyspaces of container objects (arrays, lists, maps). All considerations in 3 and 4 apply. Additionally, if your language makes no distinction between the two namespaces, your GuardingDynamicLinker should respond to dyn:setElem identically as it would to dyn:setProp.

  7. Get length of a container object

    Template: "dyn:getLength"(any-object-type)I

    Example:

    • Source code:

        a.length

    • Bytecode:

        ALOAD 2 # assume 'a' is in 2nd slot
  INVOKEDYNAMIC "dyn:getLength"(Ljava/lang/Object)I

    Returns the length of a container object. Expected to work on Java arrays, collections, and maps, as well as any other languages' container types.

  8. Create new instance

    Template: "dyn:new"(any-arguments)Ljava/lang/Object;

    Example:

    • Source code:

        new Something(any-arguments)

    • Bytecode:

        ALOAD 2 # assume 'Something' is in 2nd slot
  ... various LOADs for other arguments...
  INVOKEDYNAMIC "dyn:new"(any-arguments)Ljava/lang/Object;

    Constructs a new object. The first argument is an object that can construct new objects.

  9. Call a method on an object

    Template: "dyn:callPropWithThis:${name}"(any-arguments)any-return-type

    Example:

    • Source code:

        a.foo(args)

    • Bytecode:

        ALOAD 2 # assume 'a' is in 2nd slot
  ... various LOADs for other arguments...
  INVOKEDYNAMIC "dyn:callPropWithThis:foo"(any-arguments)any-return-type;

    Invokes a method on an object. The first argument is the object on which the method is invoked.

POJO Linker

The library ships with a linker for POJOs. Normally, when no other linker in the system can link an object, it is given to the POJO linker. The usual method invocation and property getters/setters work on any Java object as you would expect them to. Fields can be accessed as properties unless there are explicit getter or setter methods for a property of the same name as the field. Additionally, dyn:getElem, dyn:setElem, and dyn:getLength operations work on Java arrays and java.util lists and maps (dyn:getLength works on any collection).

Class and statics linking in POJO linker

Additionally, java.lang.Class objects have a virtual property named static that provides you with access to static fields and methods on that class. Static methods conforming to property getter and setter signatures are also exposed as properties on the static objects and will hide static fields of the same name. This so-called "static facet" (hey, we can only use the word "interface" for so many things) of classes is internally represented by instances of class org.dynalang.dynalink.beans.StaticClass. Currently, only public static fields and methods are accessible through the POJO linker. The library provides full transparent overloaded method resolution on property setters, methods, and constructors. You can manually access the objects exposing the static facet of classes using org.dynalang.dynalink.beans.StaticClass.forName(Class). Note that construction of instances is also a static concern of a class, and as such the POJO linker won't link a dyn:new request on java.lang.Class instances, it will instead link a dyn:new request on StaticClass instances. Think of the distinction as follows: the Java language expression java.lang.Integer maps to a StaticClass instance and will expose the constructors, the MIN_VALUE, MAX_VALUE, TYPE fields, the highestOneBit method and so on, as well as respond to dyn:new as new java.lang.Integer(1) in Java would. On the other hand, java.lang.Integer.class returns the java.lang.Class object, which is practically just a POJO, part of JVM's exposure of runtime type information, a.k.a. reflection. It does not represent the static facet of the class, and we don't want to mix the two.

As a convenience, StaticClass instances for array classes also support construction (even though these classes don't actually have constructors in JVM as their instances are created using the NEWARRAY/ANEWARRAY/MULTIANEWARRAY bytecode instructions instead of other objects' NEW) -- they expose a constructor that takes a single argument and creates an array of the expected type.

Manual overload selection

Dynalink strives to always be able to pick out the best overloaded method for the given arguments; a failure to unambiguously pick one when a human could is considered a bug (we don't know of any such case). However, sometimes, for performance reasons, a developer might want to pick out an overload themselves. In these cases, it is possible to address a Java method with specific signature, by appending the exact formal types of parameters in the method name. For example, if you know that you'll definitely be invoking System.out.println with a String argument, you can use

  dyn:callPropWithThis:println(String)

You can separate multiple argument types with commas, and use both primitive type names (int, long, etc.) and fully qualified class names; as a special case, you can use all java.lang.* classes without fully qualifying them, hence String is recognized as java.lang.String.

Advanced topic: Language runtime contexts

Some language runtimes pass "context" on stack. That is, each call site they emit will have one or more additional arguments that represent language runtime specific state at the point of invocation. This is normally thread-specific state that is accessible through a thread local too, but is more optimal when passed on stack. If you have such a language runtime, you should add the context arguments at the start of the argument list, after this but before any other arguments, and you should also make sure to invoke the setNativeContextArgCount method on the DynamicLinkerFactory to make it aware that the first few arguments in your call sites are runtime context.

In your GuardingDynamicLinker implementations, you should prepare for encountering both expected and unexpected context arguments in the link requests. If your runtime has a runtime context in the call sites, check for it, and link accordingly when you see it. It is best to pass a single runtime context argument, and make its type be a class not used for anything else, so its presence is easy to identify. If your linker is asked to link against a call site that does not expose your expected context (or your linker does not expect any runtime contexts at all), invoke LinkRequest.withoutRuntimeContext() to obtain a request with all runtime context arguments stripped and link against that. The DynamicLinker implementation is smart enough to notice that your linker returned a guarded invocation for a context-stripped link request, and will successfully link it into the call site by dropping the context arguments.

Also prepare for a situation when your linker is invoked for linking a call site that is not emitted by your own language runtime, and does not have the context arguments in the link request. You will have to make sure that your objects' methods are correctly invokable even in absence of the context -- they should be able to reacquire the context from a thread local when needed.

Odds and ends

The library contains a small amount of utilities in the org.dynalang.dynalink.support package that it itself uses, and you might find them helpful.

We specifically want to draw your attention to the Guards class that helps you create typical guards. It's conscious of class loader visibility issues and will make sure that when needed, things are only cached using weak references.

Another useful class is TypeUtilities which contains type conversion rules from Java Language Specification and similar goodies.

Yet another useful class is Lookup, which is a wrapper around MethodHandles.Lookup, and provides convenient methods such as findOwnStaticMethod. Typically, when classes need a method handle to some own static method, you need to write this:

MethodHandles.Lookup lookup = MethodHandles.lookup();
MethodHandle handle = lookup.findStatic(lookup.lookupClass(), "add", MethodType.methodType(int.class, int.class, int.class);

Compare it with:

MethodHandle handle = Lookup.findOwnStatic(MethodHandles.lookup(), "add", int.class, int.class, int.class)

Admittedly, not much of a big deal, but still much less line noise; you don't need to spell out either lookup.lookupClass or MethodType.methodType.

Add a custom footer
Add a custom sidebar
Clone this wiki locally