Understanding IOC with MEF (hot swap DLLS)

MEF  IOC
LinqPad Script: WeatherForecastR6.zip (20.2 kb) (script, Contracts project, and TableFormatterPlugin project)

This is a follow-up to my earlier IOC post, where we used the Unity Container as a second IOC alongside the system's container.  This time we'll swap Unity out for MEF (the classic System.ComponentModel.Composition) and go one step further: we'll hot swap a versioned implementation of IWeatherFormatter while the application is running.  Drop a DLL in a folder and the app picks it up; remove the DLL and the app falls back to what it had before.  No restart, no rebuild of the host.

If you haven't read the first post, the short version is this: WebApi and ASP.NET Core use a dependency injection system to instantiate classes.  When a route is selected the class for that route (HomePage, WeatherPage, or ToggleService) is instantiated and invoked, and the system looks in its service registrations (aka "container") to determine how to build the class and each of its constructor parameters: Transient (new instance each request), Scoped (one per session / request), or Singleton (everyone shares one).  As long as you never use "new" to instantiate a class (breaking the chain), you can put an interface or class in any constructor and the system will provide it.  That is Inversion of Control (IOC), aka Dependency Injection (DI).  Everything below builds on that.


Figure 1. Overview of application running - note the "Loaded formatters" list changing as the plugin DLL is dropped in and removed

A note on what you're seeing in the video: I'm able to delete (and restore) the plugin DLL while the app is running, and that is not typical MEF behavior.  It's a consequence of how this script loads the plugin.

  • Loaded from bytes (this script) - we call Assembly.Load(File.ReadAllBytes(path)).  The file is read into memory and closed, so nothing on disk is locked.  You can delete, move, or overwrite the DLL at any time; MEF just sees its catalog removed and recomposes.
  • Loaded from a path (DirectoryCatalog, AssemblyCatalog(path), Assembly.LoadFrom) - the file is memory-mapped by the process and locked.  Try to delete or replace it while the app is running and Windows will give you an "in use" error.  This is what most MEF samples do, so it's what many people expect.

Same result deployed: as long as the host loads the plugin from bytes, the file stays unlocked.  The trade off is that the loaded assembly stays in memory until the app exits (removing the DLL removes its exports, not the assembly), and restoring the file loads a second copy.  Details are in the section on loading plugin catalogs below.

Why MEF?

Unity and MEF both give us things the system container does not, but they get there differently:

  • Setter injection - the system container only does constructor injection.  With Unity you decorated a property with [Dependency]; with MEF you use [Import].  Same idea, MEF populates the property when the part is composed.
  • Change it after it's built - the system's ServiceCollection is frozen once the app is built.  With MEF we hold an AggregateCatalog, and catalogs can be added or removed at any time; the CompositionContainer sees the change and recomposes automatically.
  • Discovery - instead of registering each type by hand, MEF parts announce themselves with [Export], and we simply point the container at catalogs (a set of types, an assembly, etc.) that contain them.  That is what makes plugins practical.
  • Metadata - an export can carry information about itself ([ExportMetadata]) which we can inspect without instantiating the part.  We'll use this for the formatter "kind" (Json or Table) and its "Version".

The trade off: MEF is not a drop-in replacement for the system container.  We still use the system container to build our pages and service - MEF sits next to it, and we register the CompositionContainer as a singleton in the system container so that any page (or middleware) can ask for it in its constructor.


Figure 2.  Skeleton view of application components

Three projects, and why

The first version of this app was a single script.  To hot load something, that "something" has to live outside the script, so now there are three pieces:

  • Contracts (Contracts.csproj) - the shared types: IFoo, IWeatherFormatter, WeatherForecast, and IFormatterMetadata.
  • WeatherForecastR6.linq - the host.  It references Contracts using #:project.  JsonFormatter (and a built-in version 0 TableFormatter) are compiled right into the script.
  • TableFormatterPlugin (TableFormatterPlugin.csproj) - a separate class library containing a newer, version 1.0 TableFormatter.  It is not compiled into the script.

Why does Contracts need to be its own project?  Type identity.  If the script defined IWeatherFormatter and the plugin had its own copy, .NET (and MEF) would see two different types that happen to share a name; the plugin's export would never satisfy the host's import.  So both sides reference the same Contracts.dll.  Notice in the plugin's project file that the reference is marked Private="false"; this keeps Contracts.dll from being copied beside the plugin, so at runtime the plugin binds to the copy the host has already loaded.

The plugin builds to a dlls_staging folder - not the folder the host watches (dlls).  That way building doesn't trigger anything; you choose when to copy the DLL across, which makes for a much better demo.


Figure 3.  Contracts - the shared interfaces and metadata view

The one to notice in Figure 3 is IFormatterMetadata.  It is a "metadata view": MEF will populate its properties from the [ExportMetadata] values on each export.  Version has a [DefaultValue("0")], so an export that doesn't specify a version is treated as version 0.  Our built-in formatters don't bother to declare one.

Pages and service

The HomePage, WeatherPage, and ToggleService are built by the system container, same as before.  The difference is that where HomePage used to ask for an IUnityContainer it now asks for a CompositionContainer and uses it to resolve the IWeatherFormatter.  HomePage also receives the system's IFoo as a constructor parameter, so we can show the two containers giving two different answers.

The formatter is chosen by ResolveFormatter() (more on that below) and, just like in the Unity version, setter injection kicks in because each formatter has this property:
   [Import(AllowDefault = true)] public IFoo Bar {get;set;} 

The [Import] tells MEF to populate this property in the same manner as it would a constructor parameter.  AllowDefault = true means "if nobody exports an IFoo, leave it null rather than throwing".  This is the MEF equivalent of Unity's [Dependency].


Figure 4.  Pages and service

Below we see the results of the HomePage being clicked.  The "MEF IFoo" and "System IFoo" lines are the two containers answering the same question differently.


Figure 5. Home page

The WeatherPage does the same as the home page, but additionally lists every IWeatherFormatter that MEF currently has composed, along with the kind and version of each.  This is the "proof" page for hot swapping; watch this list as you drop the DLL in and take it out.  Before the plugin is loaded you'll see something like Json v0.0, Table v0.0; after, you'll also see Table v1.0.


Figure 6. Weather forecast page

The ToggleService flips the IsJson property and returns it (via bodyHtml) to the javascript that invoked it.  Once toggled, subsequent Home or Weather clicks will ask MEF for the other kind of formatter.


Figure 7. Toggle service

The key parts of HtmlBase, which the pages and service derive from, are unchanged from the first post.


Figure 8. HtmlBase class

The formatters

Each formatter is a MEF part.  Three attributes do all the work:

  • [Export(typeof(IWeatherFormatter))] - "I am an IWeatherFormatter, and I'm available to anyone who asks".
  • [ExportMetadata("FormatterKind", "Json")] - what kind of formatter I am.
  • [ExportMetadata("Version", "1.0")] - optional; what version I am.  Leave it off and you're version 0.


Figure 9. Formatters built into the script (json and html table v0)

Now compare to the plugin's TableFormatter.  It has the same FormatterKind as the built-in one, but declares Version 1.0.  Its caption reads "Table formatter v1.0 (plugin)" so you can tell at a glance which one rendered.


Figure 10. TableFormatter in its own project (the hot-loadable v1.0)

The built-in classes use IFoo to demonstrate how dependencies propagate, and are automagically populated, by setter injection.  Notice Bar has its own setter-injected [Import] MsgHolder Holder; that is how Bar returns "This is FooBar" - MEF composes Bar, sees it needs a MsgHolder, finds the [Export] on it, and fills it in.


Figure 11. Foo (NotBar, Bar, and MsgHolder)

Picking the winner: ResolveFormatter

With more than one Table formatter potentially composed, something needs to decide which one is used.  That is ResolveFormatter() in WebAppBuilderExtension, and it is refreshingly small:

  1. Based on IsJson, decide which kind we want ("Json" or "Table").
  2. Ask MEF for GetExports<IWeatherFormatter, IFormatterMetadata>().  This returns lazy exports - we can read the metadata without instantiating anything.
  3. Filter on the kind, order by version descending, and take the first.
  4. Only now is .Value touched, which makes MEF instantiate that one part (and setter inject it).

If nothing matches, we hand back a MissingFormatterPlaceholder - a friendly message instead of an exception.  Highest version wins.  That single rule is what makes the whole "versioned" story work: add a v1.0 and it outranks the v0; remove it and v0 is the highest remaining, so it takes over again.  Nothing is ever "unregistered" by hand.

The magic: WebAppBuilderExtension

This is where everything gets wired up.  First we build the catalogs and the container.  The built-in parts go in a TypeCatalog (an explicit list of types), and we wrap everything in an AggregateCatalog - a catalog of catalogs that we can change later.


Figure 12a. First part of WebAppBuilderExtension - catalogs and container

Next are the plugins.  We keep a dictionary of one AssemblyCatalog per DLL.  A function, syncFile, makes the catalog match what is on disk for a given file:

  • If we already have a catalog for that file, remove it from the aggregate (its exports vanish).
  • If the file no longer exists, we're done - that is the "remove the DLL" case.
  • Otherwise load the assembly, wrap it in an AssemblyCatalog, and add it to the aggregate (its exports appear).

Two details worth calling out.  First, the assembly is loaded from bytes (Assembly.Load(File.ReadAllBytes(path))) rather than from the file.  A DirectoryCatalog would lock the DLL once loaded, meaning you could not delete or move it while the app is running - which defeats the point.  Loading from bytes leaves the file free.  Second, when a file is dropped into a folder the watcher can fire before the copy is finished, so syncFile retries up to ten times, 200ms apart, and forces the types to load (cat.Parts.ToList()) so a half-copied file fails inside the retry loop instead of later.

A FileSystemWatcher on the dlls folder calls syncFile for Created, Changed, Deleted, and Renamed events; it's held in a static field so it isn't garbage collected (a collected watcher quietly stops raising events).


Figure 12b. Loading plugin catalogs from bytes and watching the folder

Then come the system container registrations, same pattern as before - the pages and service are transient, the system's own IFoo is registered as NotBar, and the MEF CompositionContainer is registered as a singleton so anything the system builds can ask for it.

Finally, the late change.  After builder.Build() we remove the NotBar catalog from the aggregate and add a catalog containing Bar.  The MEF container recomposes, and any further MEF request for an IFoo yields Bar.  The system's IFoo is still NotBar - that container can't be changed post-build.  That is the same demonstration as the Unity post, only here it is the catalog being changed rather than a registration.  The hot-loaded plugin is exactly the same mechanism, just triggered by the file system instead of by code.


Figure 12c Third part of WebAppBuilderExtension - system registrations, late catalog swap, and routes

GetHtml() below is how our pages display their content with javascript code handling button clicks and clock updates.


Figure 13.  GetHtml() code 

The middleware also shows both containers at work.  InvokeAsync receives the CompositionContainer and the system's IFoo as parameters, and writes what each one says to the console.  After the late swap you'll see "MEF Says: This is FooBar" next to "System says: Hello World".


Figure 14 - where the MiddleWare parameters are displayed

Trying it yourself

  1. Run the script and open the Weather page.  Click the toggle button until you see the table.  The caption says v0 (built-in) and the loaded list shows only v0 formatters.
  2. Build TableFormatterPlugin.csproj; the output lands in dlls_staging.
  3. Copy TableFormatterPlugin.dll into the dlls folder.  The console will report that the catalog was refreshed.
  4. Refresh the Weather page.  The loaded list now includes Table v1.0 and the caption reads v1.0 (plugin).  We never restarted anything.
  5. Delete (or move) the DLL out of the dlls folder and refresh again.  v1.0 disappears and the built-in v0 is back.


Figure 15 - before and after dropping the plugin DLL into the watched folder

A few caveats worth knowing.  Assemblies loaded from bytes can't be unloaded from the process in the default load context - removing the DLL removes its exports from MEF, but the assembly itself stays in memory until the app exits.  That's fine for a demo and for modest plugin churn; if you need true unloading, you'd look at a collectible AssemblyLoadContext.  Also, a changed DLL with the same version number is reloaded under the same rule (highest version wins, ties are arbitrary), so bump the version when you ship a new build.  Finally, MEF is the older System.ComponentModel.Composition flavor, which is what ships as a NuGet package for modern .NET; the newer System.Composition is lighter weight, but it doesn't support recomposition - which is what this demo is all about.

The decoupled nature of IOC / DI makes it easy to reuse components, because it is ultimately the container that picks the implementation for any interface.  With MEF we've pushed that one step further: the container can pick up new implementations, and newer versions of existing ones, while the application keeps on running.

Comments are closed