« Posts by layton

Virtual zip folders

So, often times when you are using a zip file, you don’t really care that it is actually a binary. You care more that it has files and folders in it; you treat it like a virtual folder. So I wrote a Java class that allows you to specify a file that has zip files as folders. Take a look!

import java.io.*;
import java.util.Deque;
import java.util.LinkedList;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.zip.ZipEntry;
import java.util.zip.ZipException;
import java.util.zip.ZipFile;
import java.util.zip.ZipInputStream;

/**
 * Allows read operations to happen transparently on a zip file, as if it were a
 * folder. Nested zips are also supported. All operations are read only.
 * Operations on a ZipReader with a path in an actual zip are expensive, so it's
 * good to keep in mind this when using the reader, you'll have to balance
 * between memory usage (caching) or CPU use (re-reading as needed).
 *
 * @author Layton Smith
 */
public class ZipReader {

    /**
     * The top level zip file, which represents the actual file on the file system.
     */
    private final File topZip;

    /**
     * The chain of Files that this file represents.
     */
    private final Deque<File> chainedPath;

    /**
     * The actual file object.
     */
    private final File file;

    /**
     * Whether or not we have to dig down into the zip, or if
     * we can use trivial file operations.
     */
    private final boolean isZipped;

    /**
     * Creates a new ZipReader object, which can be used to read from a zip
     * file, as if the zip files were simple directories. All files are checked
     * to see if they are a zip.
     * 
     * <p>{@code new ZipReader(new File("path/to/container.zip/with/nested.zip/file.txt"));}</p>
     * 
     *
     * @param file The path to the internal file. This needn't exist, according
     * to File, as the zip file won't appear as a directory to other classes.
     * This constructor will however throw a FileNotFoundException if it
     * determines that the file doesn't exist.
     */
    public ZipReader(File file){
        chainedPath = new LinkedList<File>();

        this.file = file;

        //make sure file is absolute
        file = file.getAbsoluteFile();

        //We need to walk up the parents, putting those files onto the stack which are valid Zips
        File f = file;
        chainedPath.addFirst(f); //Gotta add the file itself to the path for everything to work
        File tempTopZip = null;
        while ((f = f.getParentFile()) != null) {
            chainedPath.addFirst(f);
            try {
                //If this works, we'll know we have our top zip file. Everything else will have
                //to be in memory, so we'll start with this if we have to dig deeper.
                if (tempTopZip == null) {
                    ZipFile zf = new ZipFile(f);
                    tempTopZip = f;
                }
            } catch (ZipException ex) {
                //This is fine, it's just not a zip file
            } catch (IOException ex) {
                Logger.getLogger(ZipReader.class.getName()).log(Level.SEVERE, null, ex);
            }
        }

        //If it's not a zipped file, this will make operations easier to deal with,
        //so let's save that information
        isZipped = tempTopZip != null;
        if(isZipped){
            topZip = tempTopZip;
        } else {
            topZip = file;
        }

    }

    /**
     * Returns if this file exists or not. Note this is a non-trivial operation.
     * 
     * @return 
     */
    public boolean exists(){
        if(!topZip.exists()){
            return false; //Don't bother trying
        }
        try{
            getInputStream().close();
            return true;
        } catch(IOException e){
            return false;
        }
    }

    /**
     * Returns true if this file is read accessible. Note that if the file is a zip,
     * the permissions are checked on the topmost zip file.
     * @return 
     */
    public boolean canRead(){
        return topZip.canRead();
    }

    /**
     * Returns true if this file has write permissions. Note that if the file is nested
     * in a zip, then this will always return false
     * @return 
     */
    public boolean canWrite(){
        if(isZipped){
            return false;
        } else {
            return topZip.canWrite();
        }
    }

    /**
     * This function recurses down into a zip file, ultimately returning the InputStream for the file,
     * or throwing exceptions if it can't be found.
     */
    private InputStream getFile(Deque<File> fullChain, String zipName, final ZipInputStream zis) throws FileNotFoundException, IOException {
        ZipEntry entry;
        InputStream zipReader = new InputStream() {

            @Override
            public int read() throws IOException {
                if (zis.available() > 0) {
                    return zis.read();
                } else {
                    return -1;
                }
            }

            @Override
            public void close() throws IOException {
                zis.close();
            }
        };
        boolean isZip = false;
        while ((entry = zis.getNextEntry()) != null) {
            //This is at least a zip file
            isZip = true;
            Deque<File> chain = new LinkedList<File>(fullChain);
            File chainFile = null;
            while ((chainFile = chain.pollFirst()) != null) {
                if (chainFile.equals(new File(zipName + File.separator + entry.getName()))) {
                    //We found it. Now, chainFile is one that is in our tree
                    //We have to do some further analyzation on it
                    break;
                }
            }
            if (chainFile == null) {
                //It's not in the chain at all, which means we don't care about it at all.
                continue;
            }
            if (chain.isEmpty()) {
                //It was the last file in the chain, so no point in looking at it at all.
                //If it was a zip or not, it doesn't matter, because this is the file they
                //specified, precisely. Read it out, and return it.
                return zipReader;
            }

            //It's a single file, it's in the chain, and the chain isn't finished, so that
            //must mean it's a container (or it's being used as one, anyways). Let's attempt to recurse.

            ZipInputStream inner = new ZipInputStream(zipReader);
            return getFile(fullChain, zipName + File.separator + entry.getName(), inner);

        }
        //If we get down here, it means either we recursed into not-a-zip file, or 
        //the file was otherwise not found
        if (isZip) {
            //if this is the terminal node in the chain, it's due to a file not found.
            throw new FileNotFoundException(zipName + " could not be found!");
        } else {
            //if not, it's due to this not being a zip file
            throw new IOException(zipName + " is not a zip file!");
        }
    }

    /**
     * Returns a raw input stream for this file. If you just need the string contents,
     * it would probably be easer to use getFileContents instead, however, this method
     * is necessary for accessing binary files.
     * @return An InputStream that will read the specified file
     * @throws FileNotFoundException If the file is not found
     * @throws IOException If you specify a file that isn't a zip file as if it were a folder
     */
    public InputStream getInputStream() throws FileNotFoundException, IOException {
        if (!isZipped) {           
            return new FileInputStream(file);
        } else {            
            return getFile(chainedPath, topZip.getAbsolutePath(), new ZipInputStream(new FileInputStream(topZip)));
        }
    }

    /**
     * If the file is a simple text file, this function is your best option. It returns
     * the contents of the file as a string.
     * @return
     * @throws FileNotFoundException If the file is not found
     * @throws IOException If you specify a file that isn't a zip file as if it were a folder
     */
    public String getFileContents() throws FileNotFoundException, IOException {
        if (!isZipped) {
            return FileUtility.read(file);
        } else {            
            return getStringFromInputStream(getInputStream());
        }
    }

    /**
     * Converts an input stream into a string
     */
    private String getStringFromInputStream(InputStream is) throws IOException {
        BufferedReader din = new BufferedReader(new InputStreamReader(is));
        StringBuilder sb = new StringBuilder();
        try {
            String line;
            while ((line = din.readLine()) != null) {
                sb.append(line).append("\n");
            }
        } catch (IOException ex) {
            throw ex;
        } finally {
            try {
                is.close();
            } catch (Exception ex) {
            }
        }
        return sb.toString();
    }

    /**
     * Delegates the equals check to the underlying File object.
     * @param obj
     * @return 
     */
    @Override
    public boolean equals(Object obj) {
        if (obj == null) {
            return false;
        }
        if (getClass() != obj.getClass()) {
            return false;
        }
        final ZipReader other = (ZipReader) obj;        
        return other.file.equals(this.file);
    }

    /**
     * Delegates the hashCode to the underlying File object.
     * @return 
     */
    @Override
    public int hashCode() {
        return file.hashCode();
    }


}

Java style enums in PHP

Ever since I learned how enums worked in java, I completely fell in love with those features. PHP has a noticeable lack of enums, it only has consts and defines. Defines have the problem that they are always in global scope, and so are very prone to namespace collisions. Consts are the answer to this, but they still lack many of the features that java style enums have. So, to correct this problem, I created a class in PHP that offers all the functionality that java enums do. I call it JEnum:

abstract class JEnum{
    /**
     * Returns the name of this enum. The only difference between this method
     * and getName, is that this method can be overwritten by the extending class,
     * whereas getName is final.
     * @param integer $enum
     * @return string 
     */
    public static function toString($enum){
        self::_selfRef();
        $ref = new ReflectionClass(get_called_class());
        foreach($ref->getConstants() as $name => $constant){
            if($enum == $constant){
                return $name;
            }
        }        
        return null;
    }
    
    /**
     * Returns the name of this enum, exactly as defined in the class.
     * @param integer $enum
     * @return string 
     */
    public static final function name($enum){
        self::_selfRef();
        $ref = new ReflectionClass(get_called_class());
        foreach($ref->getConstants() as $name => $constant){
            if($enum == $constant){
                return $name;
            }
        }        
        return null;
    }
    
    /**
     * This function should be called by the class right after it is declared.
     * It will ensure that all parameters are appropriate by inspecting the
     * class reflectively.
     */
    public static final function init(){
        self::_selfRef();
        $ref = new ReflectionClass(get_called_class());
        $consts = array();
        foreach($ref->getConstants() as $constant){
            if(in_array($constant, $consts)){
                trigger_error("Multiple constants defined with the same value in ".  get_called_class(), E_USER_ERROR);
            }
            if(!is_numeric($constant)){
                trigger_error("All constants defined in ".get_called_class()." must be integer values", E_USER_ERROR);
            }
            $consts[] = $constant;
        }        
    }
    
    /**
     * Returns the name of the enum class.
     * @return string 
     */
    public static final function getDeclaringClass(){
        self::_selfRef();
        return get_called_class();
    }
    
    /**
     * Returns the numerical value of the enum, given it's name as a string
     * @param string $value The name of the enum, as a string
     * @return integer The integer value, or null, if it doesn't exist
     */
    public static function valueOf($value){
        self::_selfRef();
        $ref = new ReflectionClass(get_called_class());
        foreach($ref->getConstants() as $name => $constant){
            if($name == $value){
                return $constant;
            }
        }
        return null;
    }
    
    /**
     * This class should not be used directly, but only extended then used.
     */
    private static function _selfRef(){
        if(get_called_class() == __CLASS__){
            trigger_error("JEnum cannot be used directly", E_USER_ERROR);
        }
    }
    
    /**
     * Don't allow instantiation of the class
     */
    private final function __construct() {}
    
}

In order to use the enums, you create a new class that extends JEnum, and you define integer constants:

class TestEnum extends JEnum{
    const myEnum = 1;
    const anotherEnum = 2;
    const yetAnotherEnum = 3;
}

I have not figured out a way to automatically have the class be initialized, simply by extending the JEnum class, so the next step, after including your class is to call init on it.

include("TestEnum.php");
TestEnum::init();

This will verify that your enum follows the rules that an enum typically would: You can’t have duplicate values, and the consts can only be defined as integers. After these two steps, you can use the enums like normal constants, except now you also have the additional benefit of all the functions defined in JEnum. You can overwrite toString, but not name, just like a java enum. Here is example usage, once you create your class:

echo TestEnum::toString(TestEnum::anotherEnum);
echo "<br />";
echo TestEnum::valueOf("yetAnotherEnum");

Pivot algorithm

So today at work, I needed to write a function to create a “pivot table”, given an arbitrary number of dimensions. With a defined number of dimensions, this was easy, but making it generic took some thinking. Here’s how I solved it:


/**
* Given a set of sets, this function will return all the combinations
* of those sets. For instance, say a shirt comes in sizes S, M, and L,
* and colors Black and White, and can be labeled with either Mom or Dad,
* this will return an array of arrays with the following:
* {
* {S, Black, Mom}
* {S, Black, Dad}
* {S, White, Mom}
* {S, White, Dad}
* {M, Black, Mom}
* …
* }
* The function can accept any number of sets to combine.
*/
function pivot(array){
if(array.length == 1){
var bottom = new Array();
for(var i = 0; i < array[0].length; i++){ bottom.push(new Array(array[0][i].toString())); } return bottom; } else { var myPivot = pivot(array.slice(1)); var newArray = new Array(); for(var i = 0; i < array[0].length; i++){ for(var j = 0; j < myPivot.length; j++){ var tempArray = myPivot[j].slice(); tempArray.unshift(array[0][i].toString()); newArray.push(tempArray); } } return newArray; } } [/javascript]

Programmatic Testing

So, in my Minecraft plugin, I would like to start adding JUnit tests, but I’ve come across an issue that is preventing me from actually making many test cases. There are two issues. My plugin runs as a library inside of another plugin. I could in theory make a test harness, so it can run standalone, but for the most part, it would be nearly impossible to generate test data without it running in a real case, and writing an effective test harness that generated real data to send to the plugin, and returned valid, programmatically verifiable results would be a massive undertaking in and of itself. Secondly, many functions in the program return void, but do some action. For instance, how would one test the following function using JUnit?

public void add(int i, int j){
     System.out.println(i + " plus " + j + " is " + (i + j));
}

This would be a simple function to write a test case for, if only it returned what it computed! The first though is of course to change the function to return the result, as well as printing it to the screen. However, there are two issues: what if this function implemented an abstract function, or overwrote some parent class’s function. In this case, I cannot change the signature. Secondly, what if the results are not programmatically identifiable anyways, for instance, in Minecraft, if you send a message to a player, you would write some code similar to this:

public void sendMessage(int some_input){
     //some_input determines what player we send a message to, and what message to send
     Player p = null;
     String message = "";
     switch(some_input){
          case 1:
               p = new Player("Some player");
               message = "Some message";
           //more cases...
     }
     p.sendMessage(message);
}

In this abstract example, we could change the function to return either the player we send to, or the message we sent, but not both, at least not without doing some serious work on the function. Furthermore, this breaks the paradigm of keeping your tests separate from your production code.

I am genuinely interested in getting these test cases working, because doing regression testing when I add a new feature is starting to get incredibly cumbersome, but in many many cases in my plugin, I have issues like this, where I can’t really figure out how to best test the function, without having a whole slew of tests to manually run. It’s quickly becoming apparent to me that manually testing becomes unscalable, and that a programmatic solution is the only solution.