AspectJ load-time weaving and memory usage

To skip the background and just learn about the new weaver flags to control memory usage - just scroll to the end of this article.Weaving is a complicated process and to reduce the complexity within the weaver it likes to take control of everything and know what it is dealing with. It builds a large data structure called a World. This World knows all about types - it knows intimate details (the bytecode) of the types being woven, and it knows the basic structure of types from the classpath that are being referenced during weaving (but are not themselves to be woven).Knowing all this causes Worlds to get quite large, and they don't really give up the space they consume. This is mainly by design, a legacy design from when weaving was more something to play with than something to seriously use and rely on in your production quality application.This article is about how to solve the Worlds problems (Badum-tish!).The key data structure in the World is a typemap and the design is that it holds two categories of entity:1) a representation of a type that would be hard to recover later2) a representation of a type that would be easy to recover laterWhat falls into these two categories? From an AspectJ 1.6.4 point of view:- aspects and anything that gets woven is category (1)- types just referenced from woven types are category (2)Category (2) can be basically treated like a cache - where if the information is in memory that is great, but if it isn't we can recover it quickly. Category (1) is the data we can't recovery quickly/easily and so represents a growing fixed part of the World - as more types are woven it gets larger and the space is never released.For AspectJ 1.6.5 we are looking at what to do here. There are a few issues that with a little work could contribute a lot of benefit to the memory footprint.Firstly, it is not true that just because something gets woven that it should be considered category (1). Sometimes weaving affects the type structure (intertype declarations), sometimes it does not (basic advice weaving). If the structure of the type is not affected then we don't need to remember it in detail and once it is woven we can forget about it. If we need to recover information about it later we can do so from the classpath or from the VM that loaded it (via reflection). The fact that it was woven will no longer matter because we will only be reliant on the structure, which we know did not change due to weaving.Secondly, there are links between the two categories. Using either weak or soft references for category (2) entities is great but if there is a hard reference from a category (1) object to a category (2) object then it will never be garbage collected. Being smarter about type references will enable category (1) objects to have a 'softer grip' on the category (2) entities, allowing them to be GC'd.Some initial weaver options are live today in AspectJ 1.6.5 dev builds. These options enable the weaver to consider types only affected by advice as category (2) - disposable and recoverable.New weaver options.The new options are related to what we shall call 'type demotion'. Once enabled, then after the weaver weaves a type it will look at whether it can be demoted from the fixed set of types in the World to the garbage collectable set of types. If it can be demoted then it will be eligible for GC later.As an example, I used yourkit to attach to a weaver that had just (load time) woven 1400 classes with simple before advice. With type demotion switched off the size of the weaver was 14Meg and fixed, with no amount of forced GC's shrinking it. With type demotion switched ON, the size of the weaver was 2Meg.The options can be specified in the aop.xml file:The former option 'typeDemotion=true' switches it on. The latter option will produce a few diagnostics about demotion when it occurs. If enough people try it out and it behaves then it could become default in AspectJ1.6.5 (I am expecting some issues that need solving).I haven't fully thought through how this option will affect source compilation/weaving so use it in that scenario at your own risk (but I'd be interested in feedback!). I suspect:TypeDemotion + IncrementalCompilation = headacheThe work is being done under bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=227484 in Eclipse bugzilla.