How to compile a Java program embedded in a String? Here’s how!


Taking a small break from the mini-series on interop between languages (and also working on the main project – embedding a JVM instance in a Common Lisp image), I thought I’d share something that I frankly found amazing – how to compile and run a complete Java program contained inside a Java String! Who needs files any more, eh?

The implications of this are enormous – this means that on any machine where there is a JVM (version 6 or above) installed, even without the JDK itself*, not only can we compile and run arbitrary Java code, but also pass around Java code in strings, and compile and run them on the fly!

*Of course, it’s understood that the compiler code itself must be available in some form – as a class file, or as a Jar file.

Content

  1. A little background
  2. Implementation
    1. Compile Java code inside a String
    2. Compile Java code inside a String – fixed
  3. References

Background

I had previously written a post on how to compile Java code using the classes in the javax.tools package. That deals with the first part of the equation – how to dynamically compile Java code without resorting to brittle approaches like spawning OS processes , and without any dependency on javac.

You can find that discussion here – Dynamic Compilation of Java code (no JDK).

In this post, we will focus on the second aspect – how to store an entire Java program inside a Java String instance, and then compile and run the code.

Note: Since the javax.tools package was introduced only in Java 6, it’s only natural to deduce that these examples will work only in Java 6 (or above).

All right then, let’s get on with it!

Implementation

Top

The implementation will be in two parts. We’ll first implement a solution that appears to do the job – it can compile a Java program contained inside a string, and in many cases, even run it perfectly.

However, there is a major problem with the first implementation that we will then propose to fix by taking care of the various scenarios that might crop up in real-world use.

A first go

Top

The way we implement this solution is quite similar to the one that we used to compile Java source files. I recommend checking out the earlier post (mentioned in the Background section), but in the case of compiling a Java source string, we just have the following steps:

  • Get an instance of JavaCompiler using ToolProvider.
  • Construct the “compilation units” (which are instances of Iterable. In this specific case, we will need to construct a sub-type of SimpleJavaFileObject to pass in as the “compilation unit”.
  • Finally, create a compilation task using the compiler, and invoke call.

Well, that’s the general idea anyway. Let’s code that up now:

import java.io.File;
import java.io.IOException;

import java.util.Collections;
import java.util.Arrays;
import java.util.regex.Pattern;
import java.util.regex.Matcher;
import java.util.logging.Logger;

import java.net.URI;

import javax.tools.ToolProvider;
import javax.tools.JavaCompiler;
import javax.tools.SimpleJavaFileObject;
import javax.tools.JavaFileObject;
import javax.tools.JavaFileObject.Kind;


public enum JavaStringCompilerBasic {
	INSTANCE;
	
	private final JavaCompiler compiler;

	private Pattern namePattern;
		
	private JavaStringCompilerBasic() {
		this.compiler = ToolProvider.getSystemJavaCompiler();
		this.namePattern = Pattern.compile(".*class[ ]+([a-zA-Z0-9$_]+).*");
	}
	
	// class that defines the Java String object as a valid source file
	class EmbeddedJavaSource extends SimpleJavaFileObject {
		private String code;
		
		EmbeddedJavaSource(final String name, final String code) {
			super(URI.create("string:///" + 
							name.replace('.', File.separatorChar) + 
							Kind.SOURCE.extension),
					Kind.SOURCE);
		this.code = code;
		}
		
		@Override
		public CharSequence getCharContent(boolean ignoreEncodingErrors) {
			return this.code;
		}
	}

	/**
	  * Compile the Java code embedded inside the String.
	  * 
	  * @param program Java code
	  */	
	public void compileJavaString(final String program) {
		final String className = getClassName(program);
		
		final EmbeddedJavaSource sourceString 
				= new EmbeddedJavaSource(className, program);
		;
		
		compiler.getTask(null, null, null, null, null, 
			Collections.unmodifiableList(Arrays.asList(sourceString))).call();
	}
	
	private String getClassName(final String program) {
		final Matcher m = namePattern.matcher(program);
		
		if (m.matches() && (m.groupCount() == 1)) {
			return m.group(1);
		} else {
			throw new RuntimeException("Could not extract class name");
		}
	}

	public static void main(String[] args) {
		final String demoProgram 
			= "public class DemoProgram { " +
			  "   public static void main(String[] args) { " +
			  "       System.out.println(\"Hello from DemoProgram!\");" +
			  "    }" +
			  "}"; 	
	
		JavaStringCompilerBasic.INSTANCE.compileJavaString(demoProgram);
	}
}

As you can see, our little demo program is stored inside the demoProgram variable in the main method.


final String demoProgram
= "public class DemoProgram { " +
" public static void main(String[] args) { " +
" System.out.println(\"Hello from DemoProgram!\");" +
" }" +
"}";

Notes:: The main point of interest here is the EmbeddedJavaSource class which extends SimpleJavaFileObject. In this case, it doesn’t do much apart from creating a pseudo-URI pertaining to the fully-qualified name of the class (within the Java string).

Take it for a little spin:

Timmys-MacBook-Pro:Basic z0ltan$ javac -cp . JavaStringCompilerBasic.java 

Timmys-MacBook-Pro:Basic z0ltan$ java -cp . JavaStringCompilerBasic

Timmys-MacBook-Pro:Basic z0ltan$ java -cp . DemoProgram
Hello from DemoProgram!

Brilliant! Everything seems hunky-dory. Well, not quite. Let’s clean out the class files and try out another example to see if it is indeed working as we expect it to:

Here is the snippet we add to main:


final String problemProgram
= "package com.foo.bar;" +
"public class ProblemDemoProgram {" +
" public static void main(String[] args) {" +
" System.out.println(\"Hello from ProblemDemoProgram!\");" +
" }" +
"}";

So what’s the problem? Let’s run it:

Timmys-MacBook-Pro:Basic z0ltan$ java -cp . ProblemDemoProgram
Error: Could not find or load main class ProblemDemoProgram

Ah! That doesn’t look too good! What really happened is this – The compilation went through fine because the compiler doesn’t really care what the package definition inside the embedded Java program is (because the file doesn’t really exist on the file system now, does it?).

So it goes ahead and generates the .class file anyway. Now, when we try to run it, the java tool reads the byte code, sees a package declaration com.foo.bar, and tries to load up com/foo/bar/ProblemDemoProgram.class which, clearly, doesn’t exist!

So how do we fix it? The solution I propose it to ensure that the use case does succeed instead of failing. I propose reading the package information from the embedded Java program, parsing out the package details (if any), and creating that entire directory structure, and copying the .class file into that directory structure.

This will ensure that when java goes looking for DemoProgram.class, it really does find it, load it, and run it.

We’ll see how that solution might look like in the next section.

Final version

Top

import java.io.File;
import java.io.IOException;

import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;

import java.util.Collections;
import java.util.Arrays;
import java.util.regex.Pattern;
import java.util.regex.Matcher;
import java.util.logging.Logger;

import java.net.URI;

import javax.tools.ToolProvider;
import javax.tools.JavaCompiler;
import javax.tools.SimpleJavaFileObject;
import javax.tools.JavaFileObject;
import static javax.tools.JavaFileObject.Kind;


public enum JavaStringCompilerFinal {
	INSTANCE;
	
	private final JavaCompiler compiler;

	private Pattern namePattern;
	private Pattern pkgPattern;
		
	private JavaStringCompilerFinal() {
		this.compiler = ToolProvider.getSystemJavaCompiler();
		this.namePattern = Pattern.compile(".*class[ ]+([a-zA-Z0-9$_]+).*");
		this.pkgPattern = Pattern.compile(".*package[ ]+([a-zA-Z0-9$_.]+).*");
	}
	
	// class that defines the Java String object as a valid source file
	class EmbeddedJavaSource extends SimpleJavaFileObject {
		private String code;
		
		EmbeddedJavaSource(final String name, final String code) {
			super(URI.create("string:///" + 
							name.replace('.', File.separatorChar) + 
							Kind.SOURCE.extension),
					Kind.SOURCE);
			this.code = code;
		}
		
		@Override
		public CharSequence getCharContent(boolean ignoreEncodingErrors) {
			return this.code;
		}
	}

	/**
	  * Compile the Java code embedded inside the String.
	  * 
	  * @param program Java code
	  */	
	public void compileJavaString(final String program) {
		String className = getClassName(program);
		
		final String packagePath = getPackagePath(program);
		
		if (packagePath != null) {
			makePackagePaths(packagePath);
			className = packagePath + '.' + className;
		} 
		
		final EmbeddedJavaSource sourceString 
				= new EmbeddedJavaSource(className, program);

		compiler.getTask(null, null, null, null, null, 
				Collections.unmodifiableList(Arrays.asList(sourceString))).call();
				
		// move the compiled class into the created folder
		if (packagePath != null) {
			moveClassIntoPackagePath(className);
		}
	}
	
	private void moveClassIntoPackagePath(final String className) {
		final String sourceFile
			= className.substring(className.lastIndexOf('.') +1)
				+ ".class";

		final String targetFile 
			= className.substring(0, className.lastIndexOf('.'))
						.replace('.', File.separatorChar)
						+ File.separatorChar
						+ sourceFile;
						
		try {
			Files.move(Paths.get(sourceFile), 
					   Paths.get(targetFile), 
					   StandardCopyOption.REPLACE_EXISTING);
		} catch (IOException ex) {
			throw new RuntimeException("Error while moving file: " + sourceFile + " to "
				+ targetFile + ". Message = " + ex.getLocalizedMessage());
		}
	}
	
	private void makePackagePaths(final String pkgPath) {
		final String pkgFilePath = pkgPath.replace('.', File.separatorChar);
		
		try {
			if (!Files.exists(Paths.get(pkgFilePath))) {
				Files.createDirectories(Paths.get(pkgFilePath));
			}
		} catch (IOException ex) {
			throw new RuntimeException("Could not create directories: " +
						pkgFilePath);
		}
	}
	
	private String getPackagePath(final String program) {
		final Matcher m = pkgPattern.matcher(program);
		
		if (m.matches() && (m.groupCount() == 1)) {
			return m.group(1);
		} 
		return null;
	}
			 
	private String getClassName(final String program) {
		final Matcher m = namePattern.matcher(program);
		
		if (m.matches() && (m.groupCount() == 1)) {
			return m.group(1);
		} else {
			throw new RuntimeException("Could not extract class name");
		}
	}
	
	
	public static void main(String[] args) {
		final String demoProgram 
			= "public class DemoProgram { " +
			  "   public static void main(String[] args) { " +
			  "       System.out.println(\"Hello from DemoProgram!\");" +
			  "    }" +
			  "}"; 	
	
		JavaStringCompilerFinal.INSTANCE.compileJavaString(demoProgram);
		
		final String problemProgram
			= "package com.foo.bar;" +
			  "public class ProblemDemoProgram {" +
			  "   public static void main(String[] args) {" +
			  "      System.out.println(\"Hello from ProblemDemoProgram!\");" +
			  "  }" +
			  "}";
			  
		JavaStringCompilerFinal.INSTANCE.compileJavaString(problemProgram);
	}
}

And a sample test run with the same example as in the previous section:

Timmys-MacBook-Pro:Final z0ltan$ javac -cp . JavaStringCompilerFinal.java 

Timmys-MacBook-Pro:Final z0ltan$ java -cp . JavaStringCompilerFinal

Timmys-MacBook-Pro:Final z0ltan$ ls
DemoProgram.class					JavaStringCompilerFinal.class		com
JavaStringCompilerFinal$EmbeddedJavaSource.class	JavaStringCompilerFinal.java

Timmys-MacBook-Pro:Final z0ltan$ java -cp . DemoProgram
Hello from DemoProgram!

Timmys-MacBook-Pro:Final z0ltan$ java -cp . com.foo.bar.ProblemDemoProgram
Hello from ProblemDemoProgram!

Timmys-MacBook-Pro:Final z0ltan$ tree com/
com/
└── foo
    └── bar
        └── ProblemDemoProgram.class

2 directories, 1 file

Woo-hoo! As we can see, the directory structure is created, the .class file moved into the directory, and the code runs beautifully.

Some observations:

The only difference from the first version is that we have added extra code to create the directories (using regex to extract the relevant details from the program code), compiled the Java code stored in the strings, and copied the class file to the relevant directories. The program is then invoked normally by specifying the full class path of the Java class.

Of course, the javax.tools package is primarily aimed at compiler writers who want to create custom compilers on top of Java, but covering that use case is beyond the scope of this blog.

For instance, the better approach would have been to implement a custom JavaFileManager that then implemented the creation of the directories (if needed), or we could have used a Class Loader to load the compiled class (in case we don’t need the source, etc. The possibilities are endless!

References

Top

Here is basically the only reference that you need (really) to get started with this topic, and run with it!

Till next time then, folks!

Advertisements
How to compile a Java program embedded in a String? Here’s how!

Speak your mind!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s