Our 0.9.4 slides provide more details on the capabilities of DivConq MFT and the future direction we will take. Available for download at GitHub
This release demonstrates some of our thinking on how to handle file tasks. The scripting system is now ready to demo:
- Script triggered on successful upload
- Basic script scheduler
- Script triggered by file system events (Windows, Linux and OS X)
dcScript has been enhanced a great deal: OnError processing, File Operations, Data Streaming, GUI debugger, plus support for script libraries. A variety of example scripts are available.
Among the impressive features emerging is Streaming OpenPGP. dcScript supports an async streaming model that no existing PGP library can reasonably support, so we created our own PGP streaming. More details and screen casts to come on this all all the new features.
We are closing in on the 0.9.4 release of DivConq. In this release we added support for building the divconq libraries using POM files and “mvn install”.
Furthermore, we now have now DivConq (BETA-2) in Maven’s Central repository. This is essential for anyone trying to build Java extensions for DivConq. Many popular Java build tools, such as Maven and Gradle, require a Maven artifact when compiling. With the release of 0.9.4 all you’ll need to do is add this dependency to your build file (e.g. your POM).
If your development relies on the file system notifications you’ll also need to set the library path:
-Djava.library.path=[path to jNotify 0.94 native libs]
Our support of Maven’s Central Repository greatly simplifies the process of creating custom projects from the DivConq framework. We’ll post more on this topic in the coming months.
In the summer of 2011 I (Andy) started posting about DivConq Framework, our own little open source project. In 2011 and 2012 the focus of the framework was a Java connector to MUMPS nosql database. In 2013 my focus shifted due to customer demands and now the DivConq Framework has evolved into a File Transfer Framework.
This change is not a surprise really, since Jonathan and my professional expertise is in the file transfer industry.
Although the product is fledgling at present we believe it presents something more than a “me too”. First off it is a open source which is rare in the Enterprise class file transfer portfolio. Second the design goal is to be best of breed. Third we plan to keep it as simple as possible.
We’ll be posting more about the new Managed File Transfer (MFT) product we plan to develop so check back. In the mean time enjoy the latest demo, source code and Wiki on GitHub.
(or, “How to Walk from a Casino or Hotel on Las Vegas Avenue to McCarran International Airport”)
So you’re stuck with an afternoon flight out of Las Vegas and you don’t really want to go back to the convention? I know the feeling. In August 2014 I decided to hike from the Rio (where DEFCON 22 was held) to McCarron Airport, largely based on some TripAdvisor advice that no one should ever try this. and Michael’s Dorausch’s blog post about hiking to the Strip from the airport.
Temperature: 87 (10am) to 95 (1pm)
Weather: full sun, no wind
Distance: 3.8 miles (The Rio across Las Vegas Avenue to McCarran Airport)
Time: 3 hours (included one stop for breakfast, one long rest stop and one short water break)
Thankfully my convention was the kind that required nothing more than a change of underwear, T-shirts and sandals, so I packed light. I carried everything I needed in a backpack and carried a water bottle and a Gatorade I chilled in the hotel fridge overnight. I also wore a ballcap and a “disposable” white T-shirt I got from a vendor at the convention. (As soon as I cooled down in the terminal, I took the shirt off, threw it away and changed into a clean and dry shirt.)
FIRST LEG: Rio to Paris (20-25 minutes, 1 mile)
My first leg was pretty easy. The foot pathway along Flamingo Rd. from the Rio to the Strip is disgusting but navigable and I made it to Caesar’s Palace in about fifteen minutes. Paris is basically located kitty-corner from Caesar’s Palace so I made a quick detour through the Bellagio, crossed a couple of bridges and popped out in front of the fountain show and Paris’s outdoor cafe: Mon Ami Gabi. I had a great breakfast (tea and crepes) and explored the casino until I found the way to Bally’s.
SECOND LEG: Bally’s to Hard Rock Casino (30-35 minutes, 1.5 miles)
The second leg took me off the strip and had me wondering about the neighborhood, but there was enough haphazard development and life along the way to make me feel safe. I went east down Flamingo, then south down Koval, then east down Harmon Avenue. The Hard Rock Cafe was like an oasis and was a great place to cool down and relax.
FINAL LEG: Hard Rock Casino to McCarran Airport (25-30 minutes, 1.3 miles)
The final leg made me nervous because Google Maps didn’t have walking directions to the terminal and there were multiple advisories about heavy traffic online. However, it really was a straight and easy shot (with marked crosswalks), and it came with the bonus of walking through a cactus garden and by a an interesting statue of a cowboy. The trickiest part came when the pedestrian walkway diverted from the airport drive, but if you just follow the path you’ll come to an underpass where you can rest in the shade and drink some water (it didn’t appear to be anyone’s residence). From there, you should ignore the crosswalks to the east (they all go to dead ends) and instead go WEST (toward the cactus garden). From there all you need to do is follow the well-marked path to Terminal 1.
“Vaquero” by Luis Jiménez on the way to the terminal.
Cautions and Disclaimers
- I don’t think I’d try this at night.
- Stay hydrated.
- Walk or hike at your own risk.
- Everyone who sees you may think you gambled until you were broke! ;)
I just completed a proof of concept to access GT.M from within Java. Have a look at the readme at the project page:
Something like this could make its way into DivConq. The primary issue right now is there is no obvious (to me) way to capture the console output from GT.M via call-ins. The appeal of this approach is that it could be much faster than using SSH when connecting to a database on the local system. The nodeJS people have a project (nodem) that shows a 40x throughput over TCPIP. DivConq might be able to handle heavy loads (such as an import program) better with a direct connection like this.
I know I am not posting enough – but I am still working on the Framework a little every week. Within a month or two I expect to have something to talk about. The web layer is coming along well – web services (rpc) are well supported and so are web pages (there is even an optional CMS module – CMS engine uses jQuery Mobile for UI and it is working well).
The dcScripting language has support for some file transfer activities too. Little drops of progress have been made here and there…
I don’t typically just link to other people’s stuff in a post. But this PDF that Rob Tweed did is a must read:
“MUMPS has a pearl in its oyster called Global Persistent Variables. These are an abstraction of the B-tree structures that are normally used by MUMPS to store large volumes of data.”
As for DivConq, my schedule has been busy but I may get a chance to slip out the next release in the coming weeks. The project is not forgotten.
I’ve been quiet lately, but my day to day projects have taken the framework forward in many areas. Although 0.5.0 is not yet ready, big parts of the future versions are coming together. Here is a status update:
Version 0.5.0 (in progress)
dcDB – add support for dcSimple – Put, Get, Delete, Query (no progress)
dcDB – improve dcTables indexing, consolidate schema globals, support table instances (provide multi-homed / multi-domain support to apps) (2/3 done)
Users/Groups – improve support in task context (mostly done)
Locale Formatting – add number and date formatting to Java and MUMPS (1/2 done)
TimeZones/Chronology – add timezone support to Java and MUMPS, basic hooks for Chronology (1/2 done for US zones)
Version 0.6.0 (in progress)
dcServiceBus – support local (same JVM) service calls and schema support (done)
dcWeb – support remote service bus calls over web sockets, provide a Java client to use the web sockets interface (done)
dcClaims – provide minimal claims services – for local server or local squad. (not started)
Version 0.7.0 (in progress)
dcWeb – add HTTP RPC support for remote service bus calls. add service description language. (1/2 done)
dcWeb – provide web app framework based on dcQoo. (2/3 done)
dcAdmin – first release of admin tool, supports dcDB query and schema review (not started)
Version 0.8.0 (in progress)
dcWeb – Improve support for web frameworks, both dcQoo and dcLight. (mostly there)
dcRepository – compile developer repository, support master/project repository (not started)
At some point I need to buckle down and complete 0.5.0, but so far my user base is pushing too hard :) At this rate I would expect to see a 0.9.0 release in Q3, depending on how involved I get in Common Transfer Protocol then maybe Q4 version 0.9.5 with CTP.
Detailed in the link below are DivConq’s plans for RPC support. These plans are incomplete (there are also service discovery and service schema features) however this does cover network layer neutrality and how everything from small messages to large file transfers may route through this new protocol called Common Transfer Protocol.
Or view on SlideShare: