RSS

Monthly Archives: March 2015

Nothing’s Scarier than Imminent Success.

It’s been an interesting few weeks, but then, it’s also been an interesting few months. On the other hand, I could probably continue that line of thought, until the time span was so broad there was nothing left to hold it relative to.

The last few years have been very rough, but also fortuitous. I actually got my degree in Physics and Neuroscience, with the idea that I would someday work in neuroprosthetics. (Note to misguided Grammar Nazis: the subjects are capitalized as proper nouns when they are also unique department names. You’re welcome.) Unfortunately I also graduated in 2006, the beginning of a very dark time, and have yet to hold a physics job of any kind. The closest I’ve gotten to neuroscience was a brief stint as a substitute teacher for some elementary school special needs classes.

I basically jumped, for lack of any option, right back into programming when I got out. This isn’t to say that I haven’t had quite a few jobs that were once thought to be meant for teenagers along the way. At least five pharmacies, a hardware store, a few groceries. I am in the woeful state of having a mother who has a complete detachment from the workforce that I am confronted with; a father who really helps but has limited understanding of how one gets published today, and the limitations placed on the amount of money that I can actually make; and a brother who is nought but treacherous.

To give you an idea of what I mean by “treacherous”, I worked for him for eight months after he begged me for help on a cloud computing company that he wanted to put together, deferring applying to grad school. My sister and mother had already taken the hint and ceased contact with him, I naïvely thought that perhaps I could reunite my family through further contact and arbitration. He provided a long list of people who were supposedly behind him on it. In eight months, I could get direct contact with none of those people. Additionally, I discovered that his software patent was entirely fabricated. At the end of those eight months, he “terminated” me, I never saw a dime of the promised pay.

I quietly took note of the extent of his evil, hung up the phone, and disowned him. In the mean time, I took the GREs and got within the uncertainty limit of a perfect score on the math section. I applied to MIT, and had everything going for me, until I discovered that during that time, during my work for this play-house company, along with a not-yet-mentioned struggle with neuropathy from a medication I was on, my debt with my student loans reached a critical point. I had defaulted. My school would no longer provide transcripts. The days got even darker.

Every darkness does have its dawn, if you’re willing to work and wait long enough. I hooked up with the wonderful Sparo Vigil, here in New Mexico, and grabbed a part time job working for Home Depot. It wasn’t a high point of my life, but it made me enough to get along, and even pay back a small part of my student loans. I continued to program, working for myself, and established a sequence of frameworks to make the job easier. My cumulative experience in education and software led me to game development, and my training in the scientific method showed me a path to creating an ideal environment, and sequence of products built in that environment, with a legitimate positive change brought about in the world.

This framework is together; it works beautifully. Today, I’m going to use it for the first time, and create a finished (if not market-worthy) product. It will lift me out of the bog that I am in. I have to admit, that’s a little frightening.

If things had continued to be low, and dirty, and hard, then at least I would have a response for it. I would have a plan for how to move forward, to keep my head above the water. My ideals have been set much higher than that, though. I left the Home Depot job some time ago, maybe a year; I have Sparo to thank for keeping me afloat while I worked on the framework and my writing. Things are about to be much better, and I’m not sure how to take that.

It’s not that every little detail is finished, it’s that I’m at version 1.0. I have a system by which I can rapidly go from idea to product, possibly in an afternoon (though early on, I imagine the exception will be the rule). The rest is going to be an extensive amount of linear algebra and differential geometry, digital signal processing techniques and their implementation, modernizing of my design patterns, and writing a bridge to the Steamworks API. After that, publishing, and marketing. Later on, probably expansion of the framework interface into other languages, like Python, Ruby, and Scala. I’m looking forward to all of these things.

I’m looking forward.

I had almost forgotten how thrilling that can be.

 
Leave a comment

Posted by on March 30, 2015 in State of the Moment

 

Tags: , , ,

20 Python libraries you can’t live without

Python Tips

Hi there fellas. Today i am going to list 20 python libraries which have been a part of my toolbelt and should be a part of yours as well. So here they are

View original post 538 more words

 
Leave a comment

Posted by on March 21, 2015 in Uncategorized

 

The NIO.2 Watcher

So, I’ve been working on a side project involving the Builder tutorial. It roughly (not entirely, but roughly) works out as a machine-operated interpreter, that is, code altered by machine before being translated. After that it does something even more awesome, but it’s only capable of triggering the compilation, after alteration, through a utility that isn’t as well known as it should be.

The Watcher Utility

As of Java 7, we got the NIO.2 classes. These included Path (which most of you are probably familiar with), Files, FileSystem, asynchronous channels, and a host of other goodies. One of them was the Watch Service API.

What Watch ultimately amounts to is a device that can trigger an event any time an arbitrary subset of data is altered in some way. The easiest possible example is monitoring a directory for changes, but this is, gloriously, not exclusive. In classic Java nomenclature, one might think of it as a sort of PathEventListener, in a way; but it’s capable of a bit more than that particular name implies. It doesn’t have to be associated with Paths, and unlike most listeners, it’s less about monitoring for user-generated interrupts, and more about monitoring for system-wide circumstances, including secondary effects.

Using a Watcher

Watchers keep internal collections of keys, each one associated with a source object. This registration is typically located on the source object, at least directly. The best, and most correct, way to do this is through direct implementation of the Watchable interface. Many JDK classes, such as Path, already implement this. Once implemented, you would use the method:

Watchable.register(WatchService watchService, Kind<?>...events)

This method registers all events, of the specified types, on the provided WatchService object. Every time one of them occurs, the key is flagged as signaled, and in its own time the WatchService will retrieve the data from that key and operate.

Note that a Path can be many things. It could be a path to a directory on your machine, which is of program concern. It could be a path to a printer tray, or a server, or even a kitchen appliance (think polling the status of an automated espresso machine). In this example, I will be showing a manner in which a directory path can register to be watched for alterations.

WatchEvent.Kind interface

This can be thought of, for old school Java programmers, as the class type of a Watch event. Most of the frequently used keys are in java.nio.file.StandardWatchEventKinds, but as an interface, it is fully customizable. They only require two methods to be overridden, that is, WatchEvent.Kind.name(), which simply returns a String value representing the type of event; and WatchEvent.Kind.type(), which returns a Class that describes the location context of the event.

WatchEvent.Kind.type() may return null, and it won’t break anything; but after getting a feel for the results of StandardWatchEventKinds, you might consider implementing it. As an example, for ENTRY_CREATE, ENTRY_MODIFY, and ENTRY_DELETE, the context is a relative path between the Path being watched, and the item that has changed. (Knowing that a random item was deleted is of little if any use, without knowing which one.)

Implementing a WatchService

Most of the WatchServices you are likely to use are stock in the JDK. I’m going to start with one of them; in a later blog, I’ll probably create one from scratch, but it really is better to start simple.

For the common case of monitoring a directory, FileSystem.newWatchService() covers everything you need. It is important to get a watcher for the correct type of FileSystem, though; as many of you know, Java is capable, as of version 7, of taking advantage of the numerous file system-specific capabilities. The safest way to do it is through:

WatchService watcher = FileSystem.getDefault().newWatchService();

But there may be many points in which you intend to grab a watcher from a file system of a specific, or even custom, type. This is fine, but be aware of the extra layer of debugging.

Afterward, each path can be registered with the watch service through its Path.register(…) method. Be certain to include every variety of WatchEvent.Kind that you want to watch for. It may be tempting to simply register for every single standard type every time, but I encourage you, as a matter of practice, to consider whether you’re really concerned about each Kind before including it. They do, technically, cost a small amount of system resources; and while it may not be noticeable for small projects, when you’re dealing with massive file hierarchies it can become a concern.

When polling for changes, it is mildly more complicated than it is with Listeners. The watcher must be polled for WatchKey objects. WatchKeys are generated when a watched alteration occurs. They all have a state, which is continuously either ready, meaning the associated Watchable is valid but without events; signaled, meaning that at least one event has occurred and been registered with this WatchKey; and invalid, meaning that it is no longer sensible to consider the associated Watchable a candidate for events.

There’s more than one way to get the next signaled WatchKey, but one of the most efficient methods is WatchService.take(). This will always return a signaled WatchKey. It is a blocking method, so use it with that in mind; if no WatchKeys are yet signaled, it will wait until one is before returning.

Once you have a WatchKey, a secondary loop examines every sequential change that has occurred. (If you’re curious, if a WatchEvent occurs for a WatchKey that is already signaled, it is added to the stack and no other alterations are made; if it occurs while the WatchKey is ready, it initiates the stack and WatchKey is flipped to signaled). This is done via WatchKey.pollEvents(). For each event, you may examine the WatchEvent, and act on it accordingly.

After all is said and done, and the WatchKey has zero events left to parse, call WatchKey.reset(). This attempts to flip the WatchKey back to the ready state; if it fails (if the key is now invalid), the method returns false. This might signal, as an example, that the watched path no longer exists.

Example

Any WachService manager must be running continuously. The antipattern approach is to simply use a while-true block; but in general, it is less hazardous to make it its own thread.

import java.io.IOException;
import java.nio.*;
import java.nio.file.*;
import java.nio.file.WatchEvent.Kind;

public class DirectoryWatcher implements Runnable {
    
    private WatchService watcher;
    private volatile boolean isRunning = false;
    
    public DirectoryWatcher() throws IOException {
        watcher = FileSystems.getDefault().newWatchService();
    }
    
    /**
     * Begins watching provided path for changes.
     * 
     * @param path
     * @throws IOException 
     */
    public void register(Path path) throws IOException {
        //register the provided path with the watch service
        path.register(watcher,    StandardWatchEventKinds.ENTRY_CREATE,
                                StandardWatchEventKinds.ENTRY_MODIFY,
                                StandardWatchEventKinds.ENTRY_DELETE);
    }

    @Override
    public void run() {
        isRunning = true;
        
        while(isRunning) {
            //retrieve the next WatchKey
            try {
                WatchKey key = watcher.take();
                
                key.pollEvents().stream().forEach(event -> {
                    final Kind<?> kind = event.kind();
                    
                    if(kind != StandardWatchEventKinds.OVERFLOW) {
                        final Path path = ((WatchEvent<Path>)event).context();
                        System.out.println(kind + " event occurred on '" + path + "'");
                    }
                });
                
                if(!key.reset()) {
                    //the key should be valid now; but if it is not,
                    //then the directory was likely deleted.
                    break;
                }
                    
            } catch (InterruptedException e) {
                continue;
            }
            
            Thread.yield();
        }
    }
    
    public void stop() {
        this.isRunning = false;
    }
}

Simple enough, yes?

The register(…) method may be a little redundant; however, the run() method is where the meat is. WatchKeys are retrieved with WatchService.take(); afterward, in a parallel stream, each WatchEvent associated with that key is looped through. (When an event is of type OVERFLOW, it usually means that data on the event has been lost; not optimal, but the best course of action here is to continue to the next key.)

In this instance, the event is simply reported to the terminal, but this lambda expression is where you would take arbitrary actions according to the event. It is also possible to use an external iteration to do this, if you need to change values or perform another non-lambda-kosher action.

After all events have been iterated through, WatchKey.reset() is called, and checked. In the event that it returns false, something has happened to our directory, and the thread has become a potential resource leak; so it is shut down automatically. Otherwise, the thread then yields to other threads, and repeats itself.

Here’s a small Main class that I’ve built to use this. A single path parameter will be the directory to monitor; or it will simply watch for $HOME/watchtest.

import java.io.IOException;
import java.nio.file.Path;
import java.nio.file.Paths;

public class Main {

    public static void main(String[] args) throws IOException {
        final String location = (args.length > 0) ? args[1] : 
            System.getProperty("user.home") + "/watchtest";
        
        final Path path = Paths.get(location);
        DirectoryWatcher watcher = new DirectoryWatcher();
        watcher.register(path);
        (new Thread(watcher)).start();
        
        //wait a set amount of time, then stop the program
        try {
            Thread.sleep(10000);
        } catch (InterruptedException e) {
            //continue
        }
        watcher.stop();
    }

}

Try running it, and making a few changes to your select directory. See what it does.

And That’s It!

The next real question is how to create your own WatchService; which is totally doable. Generally, though, it isn’t necessary. The next time I come back to this subject, I’ll be going over that, possibly starting with WatchKey.Kinds. First, though, I need to get back to the project that I started this for, and I need to continue the Build Tool tutorial, so it might be a bit.

Good coding!

 
Leave a comment

Posted by on March 11, 2015 in Java, NIO.2, Programming

 

Tags: , , , , , , , , , , , , , , , , ,