Our date time formatting support starts within MUMPS. Recent releases of DivConq have considerable support for querying the database, but date time is always returned in the internal format (ISO 8601) or in the internal BigDateTime format – and also always store times using the UTC timezone. In DivConq every request to the database silently carries the current task’s (user’s) timezone (chronology) and locale. Therefore we have the info we need format a date time within MUMPS. Our current work is to code support for chronologies and timezones.

By chronology we mean Gregorian calendar, Julian calendar, Mayan calendar and such. For starters we are supporting the ISO (proleptic Gregorian) calendar. It is easy to calculate because it has a year 0 and follows the Gregorian rules back into time indefinitely. We plan to add support for some additional calendars plus add the hooks for developers to drop in support for their own custom calendars. And we plan to expand customizable calendar support to Java and Javascript. Until we do, we suggest using the Joda Time library as it is generally supported in DivConq as an alternative to BigDateTime.

Beyond picking the calendar, the chronology contains additional details such as timezone. The support we are adding for ISO (proleptic Gregorian) calendar includes support for timezones. In general the approach should work for any timezone, but our initial tests are focused on USA 2007 timezone rules. Since all dates in dcTables are stored as UTC, it makes formatting easier – there are no DST rules for UTC.

The next release of DivConq will support a minimal set of locale based date time and number formatting options.

Download : Download PDF or View on SlideShare.

The presentation covers:
1) Java Struct Parameters to MUMPS structure conversions
2) returning JSON-Like data structures
3) Error messages
4) Localized messages

Download PDF or View on SlideShare.

The presentation covers:
1) JSON Compatibility
2) Creating data structures
3) Accessing data structures
4) Use of dcSchema for data validation

Download Power Point or View on SlideShare.

The presentation covers:

1) how DivConq connects to MUMPS
2) how to create a request
3) the request-response flow
4) Java example code
5) schema example declaration
6) MUMPS example code
7) JSON and interoperability

Our open-source DivConq Framework brings a sophisticated but accessible Java library to MUMPS developers with the intention of getting the most out of the pairing. In version 0.4.5 we supplemented our powerful stored procedures feature with dcTables, a feature that aims to provide the rapid development ideal of relational databases to MUMPS. With version 0.4.8 we round out dcTables by adding support for a SELECT like command. dcTables already supported commands similar to INSERT, UPDATE and DELETE.

Part two will cover a deeper example that involves storing and reading data in a M Global – similar to how you may end up using the Stored Procedures feature your applications. In the process we’ll cover in more detail how to write the M code, how to declare the procedure in dcSchema, how to call from Java and how to process the return values.

View the M routine dctToyTest.m. Most of the code is for Stored Procedure calls, but at the top are “setup” and “cleanup” functions. This example needs setup, so before we go any further lets run that from the M prompt.

GTM>d setup^dctToyTest

GTM>

Verify that the global has been loaded, compare the code in the “setup” function with what you see in the global:

GTM>zwr ^dctData
^dctData("People",10000,"Age")=8
^dctData("People",10000,"Friends",1)=10001
^dctData("People",10000,"Friends",2)=10002
^dctData("People",10000,"Name")="Sally"
^dctData("People",10000,"Toys",1)="Legos"
^dctData("People",10000,"Toys",2)="Puzzle"
...

GTM>

There are now 6 people in our database. In addition to the records in ^dctData, there is also a ^dtcIndex which holds the indexes for the Toys and Friends fields. Sally, the example record above, has 2 friends and 2 toys. The indexes for her fields look like this:

^dctIndex("Friends",10001,10000)=1
^dctIndex("Friends",10002,10000)=1
^dctIndex("Toys","Legos",10000)=1
^dctIndex("Toys","Puzzle",10000)=1

The test data and index globals are not well designed, so the point here is just to see how the example works – not to mimic the design. When we look at the dcTables feature later the we’ll discuss design goals. That said, what we see here is that if we have a person id such as 1002 (Ginger) we can easily find who has friended that person. By looping the index – s friend=$o(^dctIndex(“Friends”,10002,friend)) – we can find all those people. Conversely, the people Ginger has friended will be in ^dctData(“People”,10000,”Friends”,n)=Friend Id.

Likewise we can find all the people who like a particular toy by looping, for example who likes Legos: s person=$o(^dctIndex(“Toys”,”Legos”,person)). If these concepts don’t make sense you need to resume review MUMPS data structures.

Lets delve into the first Stored Procedure, it just returns a list (JSON Array) of names (strings). All 6 people are listed:

GTM>d local^dcConn("QUERY dctListPeople")

Data:
:[ : Sally: Chad: Ginger: Kyle: Betty: Mike] !
Messages:
:[ ] !
GTM>

Recall that we find the M routine name in the ^Proc global.

GTM>zwr ^Proc
...
^Proc("dctListPeople")="listPeople^dctToyTest"
...

GTM>

So we should look at the “listPeople” function in “dctToyTest”. The first few lines handle parameters passed to the stored procedure. Parameters will (typically) be provided to procedures via the local variable “Params”. Parameters can be complex M structures, but typically they are just key and value pairs such as MinAge and MaxAge. When these parameters are present they limit the list of names returned so that only people within the range appear.

listPeople n id,minage,maxage
 s minage=Params("MinAge"),maxage=Params("MaxAge")
 s:minage="" minage=0
 s:maxage="" maxage=200

Note that if the parameter keys are not present we default to a value. Another option would be to log an error and return – something like this: i minage=”" d err^dcConn(90010) quit. We’ll cover error handling in a future part of this series. But it is useful to know these rules:

a) You do not have to return any data at all from a stored proc
b) If you find that something expected is missing (parameters, data) then you may log and error or warning, that message will be presented to the calling code (Java or M)
c) If no error or warning is logged then the procedure is assumed to have run correctly even if no data was returned
d) It is good practice to check for errors before returning any data, as much as possible
e) When you do want to return data from a procedure you must return it either in a List or a Record. Your structures must be well formed and syntactically correct.

Lists and Records have a definite start and end, and you must make sure they are well formed (there is always an end for every start).

 w StartList		; start list of people
 ;
 f  s id=$o(^dctData("People",id)) q:id=""  d
 . i (^dctData("People",id,"Age")<minage)!(^dctData("People",id,"Age")>maxage) q
 . ;
 . w ScalarStr_^dctData("People",id,"Name")
 ;
 w EndList		; end list of people
 ;
 quit

The start and end of the list is fairly obvious. Also by now the “for” loop should make sense. Inside the loop we first check to see if the person’s age is less than or greater than the age range provided to the procedure. If it is in the correct range we go to the next lines and write out the name.

Note that every item in a list must be preceded with a “Scalar”. Typically the Scalar is a “ScalarStr” if the following value is a string, “ScalarInt” if the following value is an integer or “ScalarDec” if the following value is a decimal.

Procedures may return complex data structures – Records in Lists, Lists in Lists, Records in Fields and Lists in Fields. Very much like JSON does. Consider the procedure “dctGetPeople” which links us to the “getPeople” function in “dctToyTest”. Here is a description of the return value for this procedure:

 [
     {
        "Id": |int, person id|,
        "Name": |string, person name|,
        "Age": |int, person age|,
        "Toys": [
	        |string, toy name|
        ],
        "Friends": [
            {
                "Name": |string, person name|,
                "Age": |int, person age|
            }
        ],
        "FriendedBy": [
            {
                "Name": |string, person name|,
                "Age": |int, person age|
            }
        ]
     }
 ]

We see that Friends are records inside of a list, the list is inside a field, the field is in a record, the record is in a list. A complex structure, but not too hard to code in M. Run it just for kicks:

GTM>d local^dcConn("QUERY dctGetPeople")

Data:
:[ :{ , Id: 10000, Name: Sally, Age: 8, Toys:[ : Legos: Puzzle] , Friends:[ :{ ,
 Name: Chad, Age: 8} :{ , Name: Ginger, Age: 9} ] , FriendedBy:[ :{ , Name: Chad
, Age: 8} ] } :{ , Id: 10001, Name: Chad, Age: 8, Toys:[ : Kite: Playdough] , Fr
iends:[ :{ , Name: Sally, Age: 8} :{ , Name: Ginger, Age: 9} ] , FriendedBy:[ :{
 , Name: Sally, Age: 8} ] } :{ , Id: 10002, Name: Ginger, Age: 9, Toys:[ : Bike:
 Softball] , Friends:[ :{ , Name: Kyle, Age: 10} :{ , Name: Mike, Age: 9} ] , Fr
iendedBy:[ :{ , Name: Sally, Age: 8} :{ , Name: Chad, Age: 8} :{ , Name: Betty,
Age: 9} ] } :{ , Id: 10003, Name: Kyle, Age: 10, Toys:[ : Softball Bat] , Friend
s:[ ] , FriendedBy:[ :{ , Name: Ginger, Age: 9} :{ , Name: Mike, Age: 9} ] } :{
, Id: 10004, Name: Betty, Age: 9, Toys:[ : Jump Rope: Softball Glove] , Friends:
[ :{ , Name: Ginger, Age: 9} :{ , Name: Mike, Age: 9} ] , FriendedBy:[ :{ , Name
: Ginger, Age: 9} ] } :{ , Id: 10005, Name: Mike, Age: 9, Toys:[ : Frisbee] , Fr
iends:[ :{ , Name: Kyle, Age: 10} ] , FriendedBy:[ :{ , Name: Ginger, Age: 9} :{
 , Name: Betty, Age: 9} ] } ] !
Messages:
:[ ] !
GTM>

Procedures do not have to have any parameters, consider the first lines of “getPeople”.

getPeople n id,tnum,fnum,fid
 w StartList		; start list of people
 ;

We jump right into writing out a list. And at the end we be sure to end our list:

 ;
 w EndList		; end list of people
 ;
 quit

The rest should fairly easily to comprehend, just do loops within loops to get at all the data you want. We do use the index global here, lets peek at that code (which is already in a loop on the records):

 . w Field_"FriendedBy"_StartList	; friended by list
 . f  s fnum=$o(^dctIndex("Friends",id,fnum)) q:fnum=""  d
 . . w StartRec	; friend info
 . . w Field_"Name"_ScalarStr_^dctData("People",fnum,"Name")
 . . w Field_"Age"_ScalarInt_^dctData("People",fnum,"Age")
 . . w EndRec	; friend info
 . w EndList					; end friended by list

Note the Start and End list for the friended by field. We just loop through the index and write out a small record (inside the List) for each friended by entry.

Lets run this via the Java Connector. Run testdb again, as you did in the Getting Connected post.

D:\dev\divconq\template>.\bin\run.bat testdb

...
option 8
...

-----------------------------------------------
   Toy Database Menu
-----------------------------------------------
0)  Main Menu
1)  List People Names
2)  List Toys
3)  List People in Tabular
4)  List People in Complex
5)  List People directly to file (Json)
6)  List People directly to file (Yaml)
7)  Add Person
1
MinAge filter [empty for no filter]: 5
MaxAge filter [empty for no filter]: 8

dctListPeople Response:
 [
        "Sally",
        "Chad"
 ]

And now for the more complex example, run option 4 and see the data coming back from the procedure formatted in JSON.

As mentioned earlier, Java calls the procedure names “dctListToys” and “dctGetPeople”. We know that the ^Proc global holds the mapping for the procedure name to the function name in M. But who sets up the mapping?

The answer is – the developer does. And so this brings us to a whole new topic in DivConq, Packages. Applications written using DivConq as assembled by combining Packages. Each Package contains Java and/or M code and adds some functionality to the overall application.

In a sense a Package is like an “add-on” only the whole application is composed of “add-ons”, separate units of functionality that may be installed or not. If you are familiar with OSGi then a package is somewhat similar in concept (we don’t use separation via class-loaders yet though, lets see what Java 8 brings). Also, to some extent, a WAR file is similar to a Package.

In any event, there is 1 core Package which all applications must have – it is “dcCore”. Combine this with your own Package “myApp” and you have all you need to make a distribution of your application. But if you choose to allow it, your customers may drop in other packages (their own customizations or third-party) on top of your install to add more functionality.

One of the great things about DivConq is that the contents of one package may override the contents of another package. Thus, if a customer dislikes images or locale translations or whatever in your application, it is fairly easy to create an override and put it in another package. This keeps their stuff separate from your distribution, and it also makes it very easy to identify the overrides during an update. You update installer can say to the user “we see you overrode X, but we changed X since your last install.” It takes work to setup that up in your installer, but the effort in checking for overrides is minimal once you know what you want to provides hints on.

This brings us to the “dcTest” package. This package would not be distributed with your application, it is just here to provide examples to developers learning DivConq. So “dcTest” keeps “dcCore” clean by keeping almost all the example files separate.

Look at your copy of the dcTest package in the Template folder: “template/packages/dcTest”. In here is the “package.xml” file that describes the package – more on that later. There is also an “m” folder that contains the M routines that dcTest relies on. Now look in “all/schema” to find the schema declarations.

Open the schema file and look for a Procedure element with Name=”dctListPeople”. Here is what that looks like:

<Procedure Name="dctListPeople" Execute="listPeople^dctToyTest">
	<Description>
		Get a list of names of all people in test data.
		Optionally add in an age range filter
	</Description>
	<RecRequest>
		<Field Name="MinAge" Type="Integer" />
		<Field Name="MaxAge" Type="Integer" />
	</RecRequest>
	<ListResponse Type="String" />
</Procedure>

Every stored procedure in DivConq needs a definition like this. The definition will be in a schema file within the Package you are developing. For example, if you are coding some Divconq tests in template then you’d probably be editing the schema file “template/packages/dctTemplate/all/schema/schema.xml”.

Stored procedures need definitions so that Java can determine that it is calling with the right parameters and that the correct data structure is being returned by the procedure. You can see in the Execute attribute how the name gets mapped to the M function.

Recall when you did “run.bat sync” and updated the M globals (option 2). The sync utility copies information about the stored procedures to M. So first you edit the schema, then you run sync, then you may call your procedure.

A future post will expand on how to make schema definitions, but for now look just at how we declare the request:

	<RecRequest>
		<Field Name="MinAge" Type="Integer" />
		<Field Name="MaxAge" Type="Integer" />
	</RecRequest>

RecRequest means we will pass in a Record (not a List) and that we’ll have MinAge and MaxAge as possible fields. So within M the Params(“MinAge”) looks like key value, but within Java/dcSchema we think of that as a record with fields. There is no Required=”True” with these fields, as such they may be absent from the call in Java.

And look at how we declare the response:

	<ListResponse Type="String" />

ListResponse (not RecResponse) means we expect a list – the Type attribute tells use it will be a list of String. Each String is a single name, of course.

This brings us to the final example in this part of the Stored Procedure series, a call in Java.

RecordStruct ages = new RecordStruct();
ages.setField("MinAge", 3);
ages.setField("MaxAge", 8);

QueryRequest lpr = new QueryRequest("dctListPeople", ages);

Although we have not yet covered RecordStruct, you can probably guess that this is what we mean when we declare the parameter as being a “RecRequest” – a record. Yes, a record with 2 fields just as declared in dcSchema. The parameter is passed into the QueryRequest object, along with the procedure name, and we have all that we need to make a call to the database.

We’ll cover more about database calls in the next part. Along with that we’ll cover error handling in stored procedures (including localization) and review a stored procedure for updating globals.

As you know, the main purpose of our open-source DivConq Framework is to let Java developers quickly harness the power of MUMPS-compatible “NoSQL” databases. To fully harness that power you need to code stored procedures in MUMPS. But sometimes you just need something *like* a relational table, and why should you have to code that?

At DivConq we say you can have your cake and eat it too. With dcDb you get the power, flexibility and speed of NoSQL when coding stored procedures in MUMPS. But, with dcTables you get the rapid development and ease of use of relational databases too. Furthermore, all your data – dcDb and dcTables – can reside in one database. Easy to administer (setup, backup and restore) and very helpful for developers. Your stored procedures can access not only your NoSQL (dcDb) data but also your dcTables data – all in the same routine using only standard MUMPS commands. For those occasions when data in NoSQL refers to data dcTables (or vice versa) nothing beats the convenience and speed of having all your data in one place.

dcTables provides Tables with Fields (columns) and Records (rows). There are methods to Insert, Update, Retire (delete), Revive (undelete) and Load records. There also also methods for loading lots of records, including a simple paging mechanism so you get only the subset of your query that you need for display. There are Select, Where and Order clauses – like SQL.

While there is a lot to like in dcTables if you know SQL, we developed dcTables to be best of breed and not a mere clone. There are differences, and mostly for the better, though every design has trade offs. Some key differences include multi-valued fields (columns), replication (eventually consistent) friendly design, and inherent data over time orientation.

It seems intuitive to say that data changes over time, for example a person’s name may change overtime. When querying a dcTable you provide a date and you get back the value(s) for that time – for 1980 you might get “Fred Smith” and for 1997 you might get “Fred Hanson” for the same record. The time orientation of data is entrenched within dcTables and encourages developers to associate data with the time period that the data is valid.

Further, dcTables are replication ready – once DivConq enables the active-active database replication feature all the data in your dcTables (as well as the operations we provide) will be ready for the migration to a replicated deployment.

Our MUMPS connection manager has also gotten a connectivity and threading face lift which should provide more reliable and efficient use of connections. Look for additional posts about the new features of dcDb and dcTables.

Edit /etc/ssh/sshd_config on the server and add (or replace) the DNS option so it looks like “UseDNS no”. Restart the SSH daemon with “service ssh restart” or equivalent. If your problems stem from DNS then your login time should go down considerably. If that doesn’t help check out the SSH FAQ.

In part two of this series I presented a review of the basics of the M programming language.

In part three of this series we examined how data is persisted in M why the M programming language is well suited to data access.

In this fourth and final part we’ll finish up some of finer points of M.

To get started you’ll want two SSH sessions. Login as a admin user (e.g. ec2-user). One of those sessions will be to the M command prompt so get to the gtmuser’s M prompt:

$ sudo su gtmuser

$ cd ~

$ ./gtmcon

The other SSH session will be used for editing M routines. So get to the gtmuser’s shell prompt in the routines directory:

$ sudo su gtmuser

$ cd ~/.fis-gtm/r/

Lets build a little program that makes a pyramid (turned on its side) in console output. We can pass in the tallest point (e.g. 4) and it will build the graphic for us. For kicks lets solve it with recursion since you may have wondered if M supports recursion (being ancient and all), well it does. So this is our expected run and output:

GTM>zl "intro1"

GTM>d pyr^intro1(4)
*
**
***
****
***
**
*

Following is a working example of how to code a solution – but this code is a little less than desirable. Here is the whole code:

intro1 ;
 ;
pyr(max) s lvl=1 d pyrr quit
 ;
pyrr ;
 f i=1:1:lvl w "*"
 w !
 q:lvl=max
 s lvl=lvl+1
 d pyrr
 s lvl=lvl-1
 f i=1:1:lvl w "*"
 w !
 ;
 quit

First, why does this work at all? The reason for that is we don’t give our variables (max and lvl) a scope. Without a scope they are “global” to that process. What I don’t like about that code is in these three lines:

 s lvl=lvl+1
 d pyrr
 s lvl=lvl-1

It works because when we call pyrr recursively the lvl is 1 greater than the last call. But lvl is not a local variable. To give our code better encapsulation, we should generally use only variables that are local to our function. A step in that direction is to change the way pyrr is called. Look at the two places “pyrr” is called below:

pyr(max) d pyrr(1) quit
 ;
pyrr(lvl) ;
 f i=1:1:lvl w "*"
 w !
 q:lvl=max
 d pyrr(lvl+1)
 f i=1:1:lvl w "*"
 w !
 ;
 quit

Making “lvl” an argument effectively makes that variable local to the function. But it is a mistake to think that our problem is completely solved.

Lets say we want our bricks to alternate between “*” and “+” for each line. A failed first attempt follows:

intro1 ;
 ;
pyr(max) d pyrr(1) quit
 ;
pyrr(lvl) ;
 d chooseBrick
 ;
 f i=1:1:lvl w brick
 w !
 q:lvl=max
 d pyrr(lvl+1)
 f i=1:1:lvl w brick
 w !
 ;
 quit
 ;
 ;
chooseBrick i lvl#2=0 s brick="+"
 e  s brick="*"
 quit

Note that in “chooseBrick” we use the variable “lvl” even though it is not an argument. This is because anytime you call a method in M the variable scope stays the same – unless you explicitly change the scope for selected variable names. Run that code and see that “lvl” works in chooseBrick. But we aren’t getting the result we hoped for.

GTM>zl "intro1"

GTM>k     ; we'll get to this command soon...

GTM>d pyr^intro1(7)
*
++
***
++++
*****
++++++
*******
******
*****
****
***
**
*

GTM>zwr
brick="*"
i=1

Our right side is all “*”. And also we see that after running we still have the variable “brick” and “i” in the process scope, because we did not give any other scope in our code. Lets try again only this time cleaning up our act. One thing we should try for is to not add variables to the process scope because their memory does not get released until the process stops. The other thing is we should try to keep scope where it belongs – this means always passing parameters and returning values when sharing variables between functions. There is a command called “new” which allows us to declare a variable to the current stack level. So we’ll use “n” and these other suggestions to make our code nice:

intro1 ;
 ;
pyr(max) d pyrr(1,max) quit
 ;
pyrr(lvl,max) n brick,i
 s brick=$$chooseBrick(lvl)
 ;
 f i=1:1:lvl w brick
 w !
 q:lvl=max
 d pyrr(lvl+1,max)
 f i=1:1:lvl w brick
 w !
 ;
 quit
 ;
 ;
chooseBrick(lvl) quit:lvl#2=0 "+"
 quit "*"
 ;

Note that we pass variables from function to function as parameters (from pyr to pyrr and from pyrr to pyrr and from pyrr to chooseBrick). Note also that we use a return value from choose brick so that we do not share the variable “brick” across functions. Every call to pyrr will result in a “new” brick variable scope, so both the left and right lines will use the correct bricks. And finally we also “new” “i”, not that it changes any functionality, but it keeps “i” out of the process scope.

Run that new code and see that it does in fact work – do the “k” before the run and the “zwr” after, there should be no variables visible.

Now there are times that you want to clear a variable but not necessarily declare it, because it has already been declared correctly. Consider these lines:

 n p
 f  s p=$o(^Toys(1,p)) q:(p="")!(p="car")

Note that we break out of the loop if we hit then end (“”) or if we come to “car”. So after that loop, “p” might be “car” or it might be “”. What if later in the code we have:

 f  s p=$o(^Toys(1,p)) q:(p="")!(p="apron")

If “apron” is in the list sometimes we would find it and sometimes not. Can you see why? If we found “car” in the code before this then we would end up looking from “car” forward and missing “apron”. So should we do “n p” again? We could, but either of these choices would be more common and use less memory:

 s p=""
 f  s p=$o(^Toys(1,p)) q:(p="")!(p="apron")

or

 k p
 f  s p=$o(^Toys(1,p)) q:(p="")!(p="apron")

A lot of M programmers would probably choose “kill” but neither is especially better than the other. Now you cannot “new” a global (one of the few language differences between local and global) but you can kill it. If you do “k ^test” all data related to that global will be cleared, it will be as if the global had never existed.

Where kill becomes really handy is the ability to “prune” branches from a M structure (global or local). Consider:

GTM>k ^test

GTM>s ^test("a")="A"

GTM>s ^test("a",1)="A1"

GTM>zwr ^test
^test("a")="A"
^test("a",1)="A1"

GTM>k ^test("a",1)

GTM>zwr ^test
^test("a")="A"

See how you could remove a whole key (subscript)? Further, any subkeys of “1″ would also be removed.

There are just a hand full of commands remaining to consider in this post, the next is “read”. To read character input from the command line use “r” and then the variable to store the result in.

 n input
 r input
 w input,!

The “if” command we have already covered in part. There is also an “else” command which is a little weird because it does not have any argument. So with “else” there are *always* two spaces after it – as in “e__”. And there is one other “got you” with “if”. In M there is only one process-wide conditional flag. If that flag changes then it will affect the outcome of the “else”.

GTM>i 1

GTM>e  w "h"

GTM>i 0

GTM>e  w "h"
h

The “else” command is not just syntactic sugar, it is a real command that can execute without any “if” nearby. Very nice for REPL and for language design, but odd for coding functions. Consider the following:

GTM>i 1 d func^ROUTINEABC

GTM>e  w "h"

Not knowing anything about “func^ROUTINEABC” what will print after the else? Will we get the “h”? At first you think, “no”. But consider that “func^ROUTINEABC” may have changed the process conditional flag. So when we get to the else we are in an uncertain state, the else may run or not depending on what happened in that function call.

To safe guard it is common practice to place an “i 1″ at the end of an “if” to put the conditional flag into the correct state. So if we are coding this it would be:

GTM>i abc d func^ROUTINEABC i 1

GTM>e  d func^ROUTINEXYZ

This way we always execute either only “ROUTINEABC” or “ROUTINEXYZ”.

The next command I’ll just introduce because it could be a whole topic, but briefly there is a “merge” command that can take two M structures and merge one of them into a branch of the other. It works with globals or locals, or across globals and locals.

GTM>k ^test

GTM>s ^test("a")="A"

GTM>zwr ^test
^test("a")="A"

GTM>s tmp(1)="A1"

GTM>zwr tmp
tmp(1)="A1"

GTM>m ^test("a")=tmp

GTM>zwr ^test
^test("a")="A"
^test("a",1)="A1"

As you can see, the contents of the variable “tmp” have been copied to the “a” branch of ^test. This works even for complex structures with many keys and sub keys. The source (e.g. “tmp”) will override any keys already in the destination (e.g. “^test”).

A built-in function to be aware of is “$d”. It returns 0, 1, 10 or 11. 0 means the key does not exist, 1 the key does exist (has value) but there are no sub keys, 10 means there are sub keys only and no value for the key, 11 means there is a value for the key and also there are sub keys.

GTM>zwr ^test
^test("a")="A"
^test("a",1)="A1"

GTM>w $d(^test("a"))
11
GTM>w $d(^test("a",1))
1
GTM>w $d(^test("a",2))
0
GTM>

That covers a lot of the day-to-day stuff well. There are two higher-level concepts I do want to cover here: transactions and locking.

Transaction support is pretty new to M, it was not part of the original design. And frankly code that is written to M design concepts cannot really use transactions in spite of the fact they are supported. The problems boils down to two things – how do you lock and what happens to your indexes when you roll back.

M is good at locking specific targets, so you can lock and index and update then unlock very quickly – the next process can then update the index and no dead lock because your first process isn’t holding on to lots of locks. With careful planning you can keep your code to a minimal of locks and lock very specific things, thus reducing conflict and preventing deadlock.

When you start using transactions in M you end up having to lock much larger swathes of data and therefore produce more blocking. M design hates inefficiency like that.

Second, consider what would happen if you did lock, update and release indexes during a transaction. What would happen if you rolled back? The index would go back to the state before you updated, which means any updates to the index after your update get wiped out. This is very bad. So really tx is just not M like and I feel should be ignored as much as possible.

Moving on to locking. Locking is a great feature of M because you can lock any key of any global with the “lock” command. The “lock” command is a little weird though as it uses a “+” to acquire and a “-” to release. Acquire:

 l +^test("a",1)

Release:

 l -^test("a",1)

Lock does not prevent another process from updating the global. So all code that updates a certain global must manually lock first, which then blocks until available, and then make the update (and then release lock). When ever possible lock only a single record at a time. Keep the record locked until all updates are made to the record as well as any index updates relating to that record. The “record first” and “1 record only” rules pretty much prevent dead locks. If you don’t write code that locks the index first and then tries to lock the record, well you won’t open the door to deadlock. You might hit the occasionally blocking wait, but that is much better than deadlock.

That wraps up the mini-series on MUMPS.