Shtting Down Stuxnet in Infected Systems

A possible approach to remove Stuxnet is to create an anti-body superior in its "resilience" in comparison to the Stuxnet malware itself.

The anti-body should be just as "infective" if not more, i.e. it must be possible to have it installed in all relevant systems on the network, in a manner similar to Stuxnet itself.

Spreading like a manner similar to Stuxnet itself, and possiblly in a more viral manner, the anti-bodies will disable Stuxnet for garbage collection.

This seems like the best approach to me when it comes to super-infections. At least that's how it happens in our bodies.

Iran has made some interesting advances in drug invention and production.

I will not be surprised to see Iran become a leader or a leading partner in the production of cyber anti-bodies that work against malware from cyber criminals.

By the way, my conjecture is that those who created the Stuxnet -- whose operations are essentially akin to biological warfare -- have most probably also created some anti-bodies.

Of course, there's an art in creating anti-bodies and in immunology.

That same art needs to be applied here.

Project Lambda and JDK 1.7

Project lambda is very cool, and developers cannot wait to see it released as part of the JDK.

Lucandra and HBasene

HBasene . . . Lucene with HBase Backend for the Distributed Index . . .

Lucandra . . . Lucene with Cassandra Backend for Distributed Index . . .

Lucandra should give the best deal when it comes to ease of administration and operation. In the long-run it should also give very good performance, with some extra partitioning plug-in work as needed.

Rough Java Implementation of GCD

Produced a rough Java implementation of Grand Central Dispatch APIs -- at this point, only the "classic" API forms, not the ones that rely on code block (or "closure") concept. JDK 6 doesn't support closures.

Lots of Tests with Casssandra

I have been doing some tests comparing the speed of EC2 "local" file manipulation vs. speed of EC2 EBS volume file manipulation. The difference is significant. I have also been doing lots of tests with Cassandra, including some with Cassandra deployed on EC2 and on EC2 with EBS, all at AWS. Interesting results. Cassandra delivers most everything it promises as far as replication, partitioning and recovery are concerned. Pluggability of partitioning and replica placement strategies are a definite plus. It would be good to have some more pluggability in data file format and composition. This will be harder but worthwhile to accomplish.

Cassandra & Distributed File System

Unlike HBase, Cassandra has no dependencies on a distributed file system. However, in a sense it implements a partitioned strategy that effectively replaces any (otherwise) required hard dependence on distributed file system capabilities.

This statement should not be taken as an endorsement of one or the other. Instead, it is simply an important architectural observation that directly impacts the overall system architecture.

Hadoop Combiners and Map-Reduce

According to the Hadoop documentation, a Hadoop Combiner can be used to speedup Hadoop Map-Reduce if the Reduce function is both commutative and associative. It would be interesting to write a program that introspects a Hadoop Reduce function to draw a conclusion regarding the validity of this constraint for the use of Hadoop Combiner but it will not be trivial.

Cloud Computing--No Single Point of Failure

A UC Berkeley technical report on cloud computing observes:

Just as large Internet service providers use multiple network providers so that failure by a single company will not take them off the air, we believe the only plausible solution to very high availability is multiple Cloud Computing providers. The high-availability computing community has long followed the mantra “no single source of failure,” yet the management of a Cloud Computing service by a single company is in fact a single point of failure. Even if the company has multiple datacenters in different geographic regions using different network providers, it may have common software infrastructure and accounting systems, or the company may even go out of business. Large customers will be reluctant to migrate to Cloud Computing without a business-continuity strategy for such situations. We believe the best chance for independent software stacks is for them to be provided by different companies, as it has been difficult for one company to justify creating and maintain two stacks in the name of software dependability

Read more of the report for other astute observations.

Design Principles for Modern Distributed Systems

The design principles used for Amazon's S3 are generally applicable to modern distributed systems design. Quoting the S3 design principles:

Amazon S3 Design Principles

The following principles of distributed system design were used to meet Amazon S3 requirements:

• Decentralization: Use fully decentralized techniques to remove scaling bottlenecks and single points of failure.

• Asynchrony: The system makes progress under all circumstances.

• Autonomy: The system is designed such that individual components can make decisions based on local information.

• Local responsibility: Each individual component is responsible for achieving its consistency; this is never the burden of its peers.

• Controlled concurrency: Operations are designed such that no or limited concurrency control is required.

• Failure tolerant: The system considers the failure of components to be a normal mode of operation, and continues operation with no or minimal interruption.

• Controlled parallelism: Abstractions used in the system are of such granularity that parallelism can be used to improve performance and robustness of recovery or the introduction of new nodes.

• Decompose into small well-understood building blocks: Do not try to provide a single service that does everything for everyone, but instead build small components that can be used as building blocks for other services.

• Symmetry: Nodes in the system are identical in terms of functionality, and require no or minimal node-specific configuration to function.

• Simplicity: The system should be made as simple as possible (but no simpler).

Elasticity in Cloud Computing

As Berkeley computer scientists have noted in a recent technical report, cloud computing (as utility) providers bring an economics where "using 1000 servers for one hour costs no more than using one server for 1000 hours." Economically, this translates into an elasticity of resources" without paying a premium for large scale ... unprecedented in the history of IT."

Interactive GAE / AWS application

Out of pure curiosity, it would be interesting to build applications with applications parts interacting with both GAE and AWS at the same time.

Building PostgreSQL on Windows

If you want to get and build PostgreSQL on Windows, try using the "git" repositories here. The repository that works with the anonymous cvs is apparently the root repository and the "git" ones are "mirrors" but it turns out that the root cvs repository has some Windows end-of-line characters that cause the build to fail. The repository you can access through "git" seems to be missing these and the build goes through. You need to ensure your cygwin installation includes readline, bison and flex.

A Brief Comparision of Concurrency in Grand Central Dispatch (GCD) and in Scala

This is a first, brief note on the general similarities and differences in how concurrency constructs in Apple's GCD (Grand Central Dispatch) compare with similar constructs in Scala.

Both Scala and GCD require Closures / Code Blocks as well as Mutability / Immutability as the language-level foundation for their concurrency models.

• Scala has supported closures (or code blocks) and mutability declarations as basic primitives in its more general functional extension to Java.
  
• Apple has proposed the addition of elements to the C language that would bring support for code blocks and mutability to the standard C language. Apple has also modified the GCC 4.2 (C compiler) to accommodate the addition of these elements. Apple has made these modifications available under the MIT license. Apple has used these extensions in OS-X 10.6.
  

Both Scala and GCD support an event driven concurrency model.

Both Scala and GCD support "segmented programming". The segmented programming model is very popular and the accepted norm in GUI programming, where the developer implements, essentially, a series of call-backs that are called in response to various GUI events. Segmented programming model is an excellent model when we wish to support concurrent execution, as is often the case with GUI development. (For more on segmented programming model, see presentation Ulf Lange and I gave in Java One 2002.)

The major difference between GCD and Scala concurrency model is that GCD relies on dispatch/queue paradigm while Scala relies on message/actor paradigm.

I believe any complexity in using either GCD or Scala has to do primarily with learning the proper use of closures and immutability.

There are a few other things that separate GCD and Scala.

Scala is a natural evolution of Java as Java was a natural evolution of C or C++. For this reason, Scala provides familiar grounds to the Java programmer. It will be very familiar to Java programmers who are willing to learn about functional features as well as more complete static typing in Scala which helps with program conciseness. (As a Java programmer, I could not put down the first Scala book I picked up. All enhancements, modifications and changes that Scala has brought with respect to Java make sense, and the resulting conciseness in Scala proves to be a blessing when it comes to development work.)

GCD relies not only on the C language extension Apple has proposed (and the C compiler changes it has made available under a very permissive licensing model) but also on the specific APIs for GCD. These APIs are particular suited to the Objective-C and Mac OS-X environment. However, the major part of GCD API can be implemented in all environments and all of it will make sense in any Unix/Linux based system.

It would be good to see Apple's proposed changes to the C language adopted by the standards committee that looks over C.

Natural Evolution ...

C-->C++-->Java-->Scala

Any, Nothing, Scala and Java

The type verification engine in Scala outdoes the one in Java.

To enable this relative strength, Scala has defined, among other features, a closed type system, with the inclusion of "Any" and "Nothing".

"Any" is the super-type of all types. This is nothing new. Java has this, too, essentially speaking.

"Nothing" is the subtype of all types. When it comes to classic and modern type theory, there's nothing new about "Nothing" either. However, "Nothing"-ness is revolutionary new when thinking of how Java has treated types.

Java designers -- perhaps for the sake of the love of trees (I mean "trees" as a graph concept) and the love of "simplicity" of type system that characterizes OS and language designers wish to serve common programmers in general -- did not stipulate "Nothing". Had they done so, we would have had a completely different language in its characteristics from what became Java.

"Nothing" is a simple change but the consequences on the type system and language features are widespread.

Hadoop Tutorial in a virtual machine

Cloudera virtual machine can be used to play with Hive and HDFS environment. See here.

Configuring Java and JDK on Snow-Leopard

You can find the best explanations and backward configurations at http://wiki.oneswarm.org/index.php/OS_X_10.6_Snow_Leopard

Who says Java is slow?

"Slowness of Java" proved to be a myth even back in 1996-1997, when Java ORBs kept beating C++ orbs in various RPC benchmarks.

Consider also, that in July 2008, Hadoop won the Terabyte Sort Benchmark: One of Yahoo's Hadoop clusters sorted 1 terabyte of data in 209 seconds. This beat the previous record of 297 seconds in the annual general purpose (Daytona) terabyte sort benchmark.

It was the first time that either a Java or an open source program had ever won.

Using SVN to find diffs on repositories

Try something like the following :

svn diff -r779768:779000

which will produce file names and diffs between those two revision numbers of a code repository under svn control.

SOA . . . Is that All ?

I originally wrote this entry on October 5, 2004, and published it on blogs.sun.com.

So, do I really get to cook SOA with these 10 ingredients? On ingredient number 6 (Governance), at least, I'd recommend another look.