MaXML 2.0 Released!

BlitzMax Forums/BlitzMax Programming/MaXML 2.0 Released!

John J.

(Posted 4 years ago) #1

(XML is a widely used and accepted format applied anywhere from word-processors to level builders)

MaXML makes it easy to load, manipulate, and save XML files. MaXML is a very easy to use library of functions for manipulating xml data, including an extremely fast xml parser, which is 16 - 736 times faster than it's competitor:

With a 1.3 MB XML file (on a Pentium 4 3.0 Ghz HT):
MaXML 2 parses in: ~290 Milliseconds
Competitor parses in: ~214840 Milliseconds

With a 50 KB XML file (on a Pentium 4 3.0 Ghz HT):
BlitzXML 2 parses in: ~10 Milliseconds
Competitor parses in: ~160 Milliseconds

New features in MaXML 2.17:
* The new xmlNode.FindChildEx() method (by Jake L.) now allows you to find nodes with specific attributes.
* The new xmlNode.SortChildrenEx() is much more flexible, allowing nodes to be sorted in almost any way you want.

New features in MaXML 2.1:
* XML files can now be loaded and saved in binary format seamlessly! (Binary files load/save about 1.5 times faster than standard XML format)
* The ability to sort nodes alphabetically based on node name, node value, attribute name, or attribute value.

New features in MaXML 2.0:
* A extremely fast, highly optimized XML parser, capable of parsing well over 4.5 MB per second (although this may vary depending on your computer's speed)
* Many new features, including node.MoveTo(), node.CopyTo(), node.Copy(), node.SortChildren(), node.SwapWith(), node.SetIndex() (for custom node ordering), etc., etc., etc.

Click here for a BlitzBasic version of MaXML

Download MaXML 2.22 Module (ZIP) - 230 KB

Included with MaXML_v2.22.zip is documentation, example code, and a Getting Started guide.

To install, simply drag the "maxml.mod" folder in the MaXML archive to your the mod/pub.mod folder under your BlitzMax install.

License: MaXML is completely free for commercial and personal use.

Haramanai

(Posted 4 years ago) #2

I am using MAXML to much right now and I felt the need to say :
Thanks
and :
Please make this thread stick.

Haramanai

(Posted 4 years ago) #3

I have and one request :
A method in the xmlDocument that will return the number of the nodes.

John J.

(Posted 4 years ago) #4

Ok. I'll try to add that in that next update.

Dreamora

(Posted 4 years ago) #5

Very usefull module.
Never really overcame my fear of using or parsing in XML till now.

Thank you :-)

Beaker

(Posted 4 years ago) #6

Have you considered adding XML sockets? Might be quite complicated to code but accessing and manipulating remote XML files is a great feature.

VP	(Posted 4 years ago) #7

Great module, takes the sting out of XML.

A word of caution to those wanting to implement XML for all their config storage or database-ish stuff: XML is hardly very lean-and-mean when it comes to storing data. Flexibility is its forte, not efficiency.

I've seen too many game config files (and in-game object descriptors) stored as external XML where an INI file style of storage would have been just as useful and half the size.

I'm all for XML being used in applications where it proves its worth (EDI being a notable perfect application), but lets keep the complexity down where possible, eh chaps? ;)

Dreamora

(Posted 4 years ago) #8

I only use it for stuff that has a hierarchy component, where the node based structure has a use.
I think too, for config files its a quite bad overhead which only makes it harder to config the file manually (which some freaky user still tend to do like me ;-))

John J.

(Posted 4 years ago) #9

vinylpusher: It's true that XML is not completely as efficient as INI or a few other formats, but I think it's flexibility and structure easily makes up for it's filesize. For example, if my .8 KB tank files in my tank game were .5 KB with INI, it really makes no difference at all for loading (both 0 millisecs - in fact, most INI parsers may even be slower than MaXML), and these days I hardly think a few extra KB's will make much of a difference. However, I definitely agree that a binary file format should usually be used if you are loading extremely large amounts of "raw" data (such as an image file, or the positions of 50,000 trees). For almost all purposes, XML is ideal (for me, at least :), but if XML just doesn't work well for your purposes, there's no point in using it, and that's fine with me.

Beaker: I haven't really considered adding XML sockets until now, but I'll see what I can do :)

VP	(Posted 4 years ago) #10

John J.: Don't get me wrong, I know that the processing overhead (thanks to your work!) is not an issue. I just want people to think before they implement :)

I know I will be using XML in an upcoming project and it is your mod I will be doing it with.

Filesize can be an issue in some cases, though you are right in saying it is not as important these days. I'm looking to the future (10-25 years down the line). Bandwidth charges will be steadily increasing and people will again begin considering every last byte!

I think there's a proposal for a binary format for storing XML. I don't know if it is close to being a standard but for the time being I think zlib compression would be a good additional option for any xml files over 2k in size.

John J.

(Posted 4 years ago) #11

I just want people to think before they implement :)

Definitely! I'm not saying that your point is wrong; I agree with you that saving file space is important. I just can't see the world going back to the days where you would say "WOW!!! 16KB RAM???"

I've been planning to impliment a binary alternate into MaXML where you can load/save a binary version of your XML files which is streamlines for speed and filesize.

Peter Scheutz

(Posted 4 years ago) #12

If size is your concern, XML files zip very well. Openoffice, opendocument etc. simly zip the xml file(s) to one package.

BlitzMax allready has the zlib , so it sould be pretty straight forward to go that way.

Nilium

(Posted 4 years ago) #13

I have to agree with JohnJ here. Being concerned about a ~2kb Xml file is really just insane now.

As for Xml itself, it's nice, but it's kindof overkill for what I do, so I invented my little SParse module. Cower.Xml is a heap o' crap now (as evidenced by JohnJ's work here). I might rewrite it, but I doubt it. Probably just use JohnJ's module instead -- beats reinventing the wheel, so to speak.

John J.

(Posted 4 years ago) #14

Update:
* Fixed bug where xmlNode.FirstChild() and xmlNode.LastChild() would crash when no node children exist
* Added ability to sort nodes based on node name, node value, attribute names, and attribute values
* Added binary XML file support for faster loading + saving

vinylpusher: Strangely, the binary format seems to be almost exactly the same size as a standard XML file. I'll probably add compression next.

VP	(Posted 4 years ago) #15

It might depend on the size of the XML file. I know zlib compression's break-even point is roughly 2K for a text file, perhaps a similar issue with the binary XML format?

Some XML files can get pretty large, like Word documents. I'm mulling the idea of a sales/invoicing/inventory system which outputs Word compatible XML when you want to generate an invoice or delivery note. The generated XML files need to be fed into a database, so I am concerned about size. A simple, single-page Word XML file is 23KB. Multi-page invoices with 100's of line items is probably going to push half a meg. Multiply that by a few hundred thousand over the lifetime of the system and you start to fill hard drives (and slow the database down).

Zlib would work wonders :)

Jay Kyburz

(Posted 4 years ago) #16

Hey John, I was just installing this on my pc and knoticed there is no entry for maxml in the code archives section of the website.

Booticus

(Posted 4 years ago) #17

John thanks again for the module. Its very neat!

morszeck

(Posted 4 years ago) #18

It's a cool Tool !!! This simplify me some. Because it's very fast also.

Something can add in addition:

mynode.SetAttribute( "name", "value" )

Because,

mynode.Attribute( "name" )

if "name" is not available, so automatically a "name".

John J.

(Posted 4 years ago) #19

Hey John, I was just installing this on my pc and knoticed there is no entry for maxml in the code archives section of the website.

I didn't put it in the code archives because I would be declaring it public domain. If I submitted it to the code archives under public domain, would I still own the copyright to MaXML?

morszeck: I'm not quite sure I understand. Is there a feature you want me to add? To add an attribute, just do this:

mynode.Attribute("name").Value = "value"

And to read an attribute, you can do this:

value$ = mynode.Attribute("name").Value

And if you want to detect when an attribute does not exist, you can do this:

If mynode.HasAttribute("name") = False Then Throw "Cannot find ~qname~q attribute!"

morszeck

(Posted 4 years ago) #20

When the root is:
<test>

now, i write:

root = mydoc.Root()
root.Attribute("name")

then becomes automatically:

<test name="">

Is this so meant?

John J.

(Posted 4 years ago) #21

Yes, that's the way it's supposed to work. Attribute() is meant as an easy way to read or write node attributes. However, the statement "root.Attribute("name")" on it's own is no use, really, unless you do something like "root.Attribute("name").Value = ...".

If you still end up with alot of "blank" attributes like that, you can always call node.CleanAttributes(), which removes them for you.

morszeck

(Posted 4 years ago) #22

Ok, thx... but the error messages must be still improved. Get time to time errors from Linker. Without line number or so ...

John J.

(Posted 4 years ago) #23

Get time to time errors from Linker.

That's strange. Could you list some of the errors you're getting?

morszeck

(Posted 4 years ago) #24

This is the error message:

Building xmldoc_creater
Compiling:xmldoc_creater.bmx
flat assembler version 1.64
3 passes, 21325 bytes.
Linking:xmldoc_creater.debug.exe
C:/Programme/BlitzMax/bin/ld.exe: cannot find C:/Programme/BlitzMax/mod/_n = root_n.mod/addnode(
Build Error: Failed to link C:/Dokumente und Einstellungen/rene/Desktop/BlitzMax_Editor_2/xmldoc_creater.debug.exe
Process complete

// Edit //

It's a Bmax Bug. See here: http://www.blitzbasic.com/Community/posts.php?topic=57112

A variable name "import_n" in a Method create an linker error!!!

John J.

(Posted 4 years ago) #25

That's weird - "import_n" causes a linker error for me too (but I'm not using the latest BlitzMax version, so I don't know if it's fixed already or not). MaXML doesn't use any variables called "import_n", so at least MaXML is safe for now.

John J.

(Posted 4 years ago) #26

Update: Version 2.15 includes some misc. bug fixes (escape codes were not being loaded/saved properly).

MadMunky

(Posted 4 years ago) #27

Is there a version out for Blitzmax 1.18 yet or any way of getting it working?

Why is it every bmax update u have to redownload all mods its a pain!

John J.

(Posted 4 years ago) #28

I'll upload a re-compile after downloading BlitzMax 1.18, which should hopefully fix this.

P.S. Until then, you could re-compile it yourself. Just type "bmk makemods pub.maxml" at a command prompt.

FlameDuck

(Posted 4 years ago) #29

Couldn't you talk to skidracer about getting this added to the axe repository?

MadMunky

(Posted 4 years ago) #30

i tryed to re-compile it but still doesnt work :(

drnmr

(Posted 4 years ago) #31

mine says "can't find interface for module 'pub.maxml'"
i'm using bmax 1.18.

AntMan - Banned in the line of duty.

(Posted 4 years ago) #32

You can strip the module lines and just import it into your project manually. just copy the source from the mod folder to your project and type 'import "MaXml.bb"

John J.

(Posted 4 years ago) #33

I'm trying to update MaXML, but I'm having trouble downloading BlitzMax 1.18 (the download keeps terminating early causing a corrupted exe) - probably because of my slow dial-up connection.

I'll keep trying.

drnmr

(Posted 4 years ago) #34

aurora- ???

John J.

(Posted 4 years ago) #35

Aurora means that you can import it as a source file, instead of a module (if you remove or comment out the Module and ModuleInfo lines)

Anyway, I finially got the latest BMax version, so hopefully I'll have a working version up soon.

John J.

(Posted 4 years ago) #36

Update: The latest version of MaXML is now compatible with BlitzMax 1.18.

Tachyon

(Posted 4 years ago) #37

I would really enjoy seeing some more examples (sample code) of this XML module in action. Does anyone feel generous enough to a few bits of code, pretty please?? :) Thanks!

Sean Doherty

(Posted 4 years ago) #38

John,

I'm trying MaXML, but if I load my file over and over I eventually get an error or odd results. Is there any bugs I should be aware of? Here is my module:

	Method LoadXmlMap:TSector(sMapName:String, pTGamePlayManager:TGamePlayScreen) 
		
		Try
			'Create a new XML document object and load data from the example.xml file.
			Local pXmlDocument:xmlDocument = Null  
			Local pHeaderXmlNode:xmlNode   = Null           
			Local pSectorXmlNode:xmlNode   = Null
			Local pStarshipXmlNode:xmlNode = Null  
			Local sTitle:String  
			Local sAuthor:String 
			Local sDescription:String 
			Local iSectorOID:Int  
			Local sSectorName:String    
			Local iSectorX:Int, iSectorY:Int
			Local iStarshipX:Int, iStarshipY:Int 
			Local iHullOID:Int, iPlayerOID:Int 
			Local pTSector:TSector                                   = Null
			Local pTPlayer:TPlayer                                   = Null 
			Local pTShieldIndicator:TShieldIndicator                 = Null   
			Local pTStarship:TStarship                               = Null 
			Local pTHull:THull                                       = Null 
			Local pTVehicleFlockingStrategy:TVehicleFlockingStrategy = Null

			pXmlDocument = New xmlDocument 

			'Load "example1.xml" into doc1
			pXmlDocument.Load("xml/" + sMapName)

			pHeaderXmlNode = pXmlDocument.Root().FindChild("Header")		
			sTitle = pHeaderXmlNode.FindChild("Title").Value
			sAuthor = pHeaderXmlNode.FindChild("Author").Value
			sDescription = pHeaderXmlNode.FindChild("Description").Value
            
			'-------------------------------------------
			'Find the node that contains the XML Sector.
			'-------------------------------------------
			pSectorXmlNode = pXmlDocument.Root().FindChild("Sector")
			
			While pSectorXmlNode <> Null
				If pSectorXmlNode.name = "Sector" Then
				
					iSectorOID = Int(pSectorXmlNode.FindChild("OID").Value)
					Print "Reading XML SectorOID = " + iSectorOID
					
					sSectorName = pSectorXmlNode.FindChild("Name").Value   
					iSectorX = Int(pSectorXmlNode.FindChild("X").Value)
					iSectorY = Int(pSectorXmlNode.FindChild("Y").Value)
					
					'Create the Sector.			
					pTSector = New TSector  
					pTSector.Initialize(pTGamePlayManager.m_pTGalaxy, iSectorOID)
					pTSector.SetPosition(iSectorX, iSectorY)

					'Get the Next Node
					pStarshipXmlNode = pSectorXmlNode.FindChild("Starship")
					
					While pStarshipXmlNode <> Null 
					
						Print "Loading Starship"
						iStarshipX = Int(pStarshipXmlNode.FindChild("X").Value)	
						iStarshipY = Int(pStarshipXmlNode.FindChild("Y").Value)	
						iHullOID   = Int(pStarshipXmlNode.FindChild("HullOID").Value)	
						iPlayerOID = Int(pStarshipXmlNode.FindChild("PlayerOID").Value)	
	
						'Create the Starship.
						pTShieldIndicator = TEntityFactory.GetInstance().CreateShieldIndicator()
						pTHull = TComponentFactory.GetInstance().GetHull(iHullOID).Clone()
						pTPlayer = pTGamePlayManager.GetPlayer(iPlayerOID) 						
						pTStarship = TEntityFactory.GetInstance().CreateStarship(iStarshipX, iStarshipY, iHullOID, pTPlayer, pTShieldIndicator, pTHull, pTSector)
						pTStarship.Launch()
	
						'Create the Starship Strategy.
						pTVehicleFlockingStrategy = New TVehicleFlockingStrategy
						pTVehicleFlockingStrategy.Initialize(pTStarship)
						pTStarship.SetStrategy(pTVehicleFlockingStrategy)	
						
						pStarshipXmlNode = pStarshipXmlNode.NextSibling()	'Get the next node	 
						If pStarshipXmlNode <> Null
							Print pStarshipXmlNode.name
						End If
					Wend
						
				End If
		
				pSectorXmlNode = pSectorXmlNode.NextSibling()	'Get the next node   
			Wend
			
			'Close the xml file by freeing the xml document
			pXmlDocument = Null

			GCCollect
		     		     
	    Catch ex:Object
			Throw("TLevelBuilder.LoadXmlLevel - " + ex.ToString())
		End Try
		
	End Method

Works must of the time?

Sean Doherty

(Posted 4 years ago) #39

XML File:

<?xml version="1.0"?>
<StarchonMap>
   <Header>
      <Title>Foot Hold</Title>
      <Author>Sean Doherty</Author>
      <Description>The Bio Race has attached a Common Wealth Member and established a Base of operations.</Description>
   </Header>
   <Sector>
      <OID>10</OID>
      <Name>Toga Prime</Name>
      <X>800</X>
      <Y>300</Y>
	<Starship>
	   <X>0</X>
	   <Y>0</Y>
	   <HullOID>2</HullOID>
	   <PlayerOID>2</PlayerOID>
	</Starship>
	<Starship>
	   <X>0</X>
	   <Y>0</Y>
	   <HullOID>2</HullOID>
	   <PlayerOID>1</PlayerOID>
	</Starship>
	<Starship>
	   <X>0</X>
	   <Y>0</Y>
	   <HullOID>2</HullOID>
	   <PlayerOID>1</PlayerOID>
	</Starship>
   </Sector>
   <Sector>
      <OID>11</OID>
      <Name>Earth</Name>
      <X>800</X>
      <Y>500</Y>
	<Starship>
	   <X>0</X>
	   <Y>0</Y>
	   <HullOID>2</HullOID>
	   <PlayerOID>2</PlayerOID>
	</Starship>
   </Sector>
</StarchonMap>

Sean Doherty

(Posted 4 years ago) #40

John,

I put some test code in place and it seems that on a fairly regular basis the following lines return a null node when they should not:

pStarshipXmlNode = pSectorXmlNode.FindChild("Starship")

and

pSectorXmlNode = pSectorXmlNode.NextSibling()

Any idea what I'm doing wrong?

John J.

(Posted 4 years ago) #41

Your code seems to work fine for me. I stepped through the entire load process and it seemed to process every node properly. I also ran it several times in the same program, and the output was the same:

Reading XML SectorOID = 10
Loading Starship
Starship
Loading Starship
Starship
Loading Starship
Reading XML SectorOID = 11
Loading Starship

If what you are describing is a MaXML bug, I will try to fix it as soon as possible, but until I can reproduce the problem, I doubt I will get very far.

Jake L.

(Posted 4 years ago) #42

Hi John,

I "wrote" a function FindChildEx, which not only looks for a given node-name but for a matching attribute as well. Example: mydata:xmlnode=FindChildEx ("playerdata","name","Jake")

Maybe you want to add this to your official mod.

Here's the code:

Method FindChildEx:xmlNode(Name:String, AttrName$,AttrValue$,Recurse = True, MatchCase = True)
		Local Node:xmlNode, temp:xmlNode
		
		If ChildList.IsEmpty() Then Return Null
		
		If MatchCase Then
			For Node = EachIn ChildList
				If node.Name = Name and node.Attribute(AttrName).Value=AttrValue Then Return Node
			Next
		Else
			Name = Name.ToLower()
			For Node = EachIn ChildList
				If node.Name.ToLower() = Name  and node.Attribute(AttrName).Value=AttrValue Then Return Node
			Next
		End If
		
		If Recurse Then
			For Node = EachIn ChildList
				If node.ChildList.IsEmpty() = False Then
					temp = node.FindChildEx(Name, AttrName, AttrValue,Recurse, MatchCase)
					If temp <> Null Then Return temp
				End If
			Next
		End If
		
		Return Null
	End Method

I have a hard time to modify your SortNode code to just sort children of a given nodename. This could be useful as well (I have a node <typedef> with children-nodes <field> and <function> and would like to sort the fields as well as the functions). Any idea?

Anyway, you should know that I dropped any .ini/registry-stuff since using your lib - it's easy to use and works like a charm.

John J.

(Posted 4 years ago) #43

Thanks for the code for FindChildEx() - it looks useful. I'll include it with the official version of MaXML soon.

I have a hard time to modify your SortNode code to just sort children of a given nodename.

I probably need to re-write that sort code - it's a little messy. Anyway, I'll see what I can do to add that feature to MaXML. Do you just want the ability to sort nodes of a specific name?

Jake L.

(Posted 4 years ago) #44

In detail, sorting childs (of just a given name) by an attribute or value.

Example (using a node's value here, but could be attribute as well):

<typedef>
<function>ASome</function>
<function>CSome</function>
<global>CDSome</global>
<function>BSome</function>
<global>ASome</global>
<global>BSome</global>
</typedef>

Now, the end result should look like:

<typedef>
<global>ASome</global>
<global>BSome</global>
<global>CDSome</global>
<function>ASome</function>
<function>BSome</function>
<function>CSome</function>
</typedef>

To do this, I first need to sort by nodename (SortChildren can do this atm) and then each "subset" (<global> and <function>) by it's value. Maybe you could enhance SortChildren to sort by attribute/value just a given subset (=nodename).

John J.

(Posted 4 years ago) #45

How about adding the ability to do this:

Node.SortChildren(SORTBY_NODE_NAME + SORTBY_NODE_VALUE)

(sort of like Blitz3D's entity flags)
This way, you could sort nodes any way you want very easily.

Jake L.

(Posted 4 years ago) #46

Sounds very good! Can't wait for the next version ;)

John J.

(Posted 4 years ago) #47

New features in MaXML 2.17:
* The new xmlNode.FindChildEx() method (by Jake L.) now allows you to find nodes with specific attributes.
* The new xmlNode.SortChildrenEx() is much more flexible, allowing nodes to be sorted in almost any way you want.

Sean Doherty

(Posted 4 years ago) #48

John,

Did you find the issue I reported?

Thanks

John J.

(Posted 4 years ago) #49

So far I haven't been able to reproduce the issue. If you could send me some example code (along with an xml file, if possible) which demonstrates the problem, I would be able to fix it much quicker.

JonasL

(Posted 4 years ago) #50

MaXML is a really nice module. Thanks for providing it. I have downloaded the version 2.17.

On MacOS X I had to "Build Modules" to make it work and after that it works just fine.

On my WinXP machine, however, I get an "Unhandled Exception: Illegal bank offset", when running both example programs. I guess there is some problem with the build and the latest version of BlitzMax.

I just wanted to report it. I'm no BMX wizard, but I guess the problem can be solved by building the modules. However, in Win the menu items are disabled, so I guess I need to install something first. Guess I need to hit the documentation and the forums to find out. ;)

John J.

(Posted 4 years ago) #51

JonasL: Until I release a re-compiled version, you can re-make the module yourself by going to MS-DOS command prompt and entering the following:

"cd C:\Program Files\BlitzMax\Bin"
"bmk makemods pub.maxml"
"docmods"

... then everything should be up to date.

JonasL

(Posted 4 years ago) #52

John J: Thanks. It now works like a charm in Win as well. Great work with this module. Now I can continue working with our new game Stray Hero (sorry, no info on the web site yet). I really wanted XML support for data files to make them more editable.

John J.

(Posted 4 years ago) #53

Update: MaXML 2.18 fixes a bug where Unix-style return characters in a file causes an error (also, MaXML 2.18 is recompiled with the latest version of BlitzMax).

Gabriel

(Posted 4 years ago) #54

I'm getting problems with this in Release mode. I have a loop which loops through all the nodes which are children of the root node.

   Local node:xmlNode=root.FirstChild()
   While node <> Null
      If node.Name = "ImagePackImage" Then
         ' DO STUFF
      Else
         Debug_Log "INVALID NODE TYPE"+node.Name
      End If
      node = node.NextSibling()
      If node=Null
         Debug_Log "FOUND LAST NODE"
      End If
   Wend

In debug mode, it works fine and finds every node. In release mode, it stops finding nodes long before it finds the last one. It does not error in my code ( verified because "FOUND LAST NODE" is written to my custom debuglog ( since there is no Blitz debuglog in Release mode )

Any idea why NextSibling might start returning a null node in release mode and not in debug? I've checked the XML file and it's great. It was also created with your module.

EDIT: I'm debugging your module as best I can, and will post more info as I find it.

Specifically, NextSibling is returning Null because it believes the node I pass to it does not have a parent, which of course it must or it would never have been returned as a sibling the time before :/

EDIT2: I've tried rewriting my code to use ChildCount() and GetChild() but that works perfectly in debug mode and just MAV's in Release mode. Presumably for a similar or the same reason.

Gabriel

(Posted 4 years ago) #55

Just to confirm, it's definitely a problem with MaXML because I've just changed my modules and code to use LibXML instead and the problems vanish with LibXML.

John J.

(Posted 4 years ago) #56

Thanks for letting me know about this. I'll try to fix it ASAP.

John J.

(Posted 4 years ago) #57

Unfortunately, I'm still having trouble finding the cause of the debug/release problem. I'm beginning to think it's a problem with Blitzmax, since it works perfectly in debug mode, and everything operates as I would expect it to, but in release it crashed for no apparent reason. At least, I'm reasonably certain that no compiler should ever have major behavioral differences between debug and release like this.

Just to confirm, it's definitely a problem with MaXML because I've just changed my modules and code to use LibXML instead and the problems vanish with LibXML.

Maybe the problems vanish with libxml because libxml was written in C, not BlitzMax (the libxml module for BlitzMax is simply a wrapper, which allows access to the C code through BlitzMax). Unfortunately, I may eventually be forced to port MaXML to C++ (since I have never encountered major debug/release inconsistancies with C/C++). Then, I would make a wrapper in BlitzMax so you could use it in BlitzMax. Hopefully, I won't have to resort to this, however.

Gabriel

(Posted 4 years ago) #58

Yes sorry, when I said it's definitely a problem with MaXML, I meant as opposed to my use of it, not as opposed to BlitzMax itself. I did have a good look through your code and I couldn't find anything either, so you may well be right that it's a BlitzMax issue or bug.

As you say, the only differences between debug and release should be bounds checking and things like that. There shouldn't be any behavioural differences, so it's indeed very odd.

skidracer

(Posted 4 years ago) #59

How do I go about reproducing this bug?

Gabriel

(Posted 4 years ago) #60

I've since rewritten all my code to use LibXML, so I don't have the example handy any more. I've got a lot on today, but I'll try to cobble together a quick sample and include the XML file to go with it. Hopefully tomorrow. I really have no idea whether or not the problem lies with BlitzMax or MaXML though.

John J.

(Posted 4 years ago) #61

skidracer:
Here is how to reproduce the problem I encountered when trying to run one of the MaXML examples in release mode:

Software:
BlitzMax Version: 1.20 (latest)
MaXML Version: 2.18 (latest)

Test code:

Import pub.MaXML

Strict

Local node:xmlNode, root:xmlNode
Local name:String
Local foundSomething = False

Local XMLFile:xmlDocument = xmlDocument.Create("example2.xml")	
root = XMLFile.Root()

root.SortChildren(SORTBY_ATTR_VALUE)

Example2.xml contents:

<?xml version="1.0"?>
<addressbook title="XML Address Book Example">
  <person firstname="Bob" lastname="Johnson">
    <phone>456-6784</phone>
  </person>
  <person firstname="Joe" lastname="Roberts">
    <phone>155-8776</phone>
  </person>
  <person firstname="Bill" lastname="Baker">
    <phone>955-3744</phone>
  </person>
  <person firstname="Robert" lastname="Jones">
    <phone>153-4487</phone>
  </person>
  <person firstname="Phillip" lastname="Taginski">
    <phone>205-0986</phone>
  </person>
  <person firstname="George" lastname="Smith">
    <phone>577-2264</phone>
  </person>
    <phone>754-2908</phone>
  <person firstname="Luke" lastname="Jackson">
    <phone>123-4567</phone>
  </person>
  <person firstname="Guy" lastname="Wilson">
    <phone>131-1313</phone>
  </person>
  <person firstname="Alan" lastname="Drake">
    <phone>332-9179</phone>
  </person>
</addressbook>

Debug mode output:

Building test
Compiling:test.bmx
flat assembler version 1.64
3 passes, 2756 bytes.
Linking:test.debug.exe
Executing:test.debug.exe

Process complete

Release mode output:

Building test
Compiling:test.bmx
flat assembler version 1.64
3 passes, 1599 bytes.
Linking:test.exe
Executing:test.exe
Unhandled Memory Exception Error
Process complete

skidracer

(Posted 4 years ago) #62

Yikes,

The problem with that code is XMLFile has been garbage collected. Adding a reference to the end of the problem code stops the crash:

Import "MaXML.bmx"

Strict

Local node:xmlNode, root:xmlNode
Local name:String
Local foundSomething = False

Local XMLFile:xmlDocument = xmlDocument.Create("example2.xml")	
root = XMLFile.Root()


root.SortChildren(SORTBY_ATTR_VALUE)

root = XMLFile.Root()

So what is happening?

When a List goes out of scope the _head link must be cleared (nulled), so the your node.sort method I think is crashing due to the unexpected destruction of the document from under it and hence the unexpected nulling of the _head member. Short of making _head private we're bashing our heads on a rational fix for this.

The reason the above only crashes in Release mode is that unfortunately garbage collection is a lot more aggressive in release where temporary objects may be held for shorter times in registers instead of the stack and hence they can get collected earlier.

For a temporaty fix for MaXML you could add to xmlNode:

Field _Owner:xmlDocument

and in the RootNode creation add:

_RootNode._Owner=Self

Getting rid of the _RootNode and having xmlDocument extend xmlNode also seems to work well and doesn't create the problematic circular dependency that the above does.

John J.

(Posted 4 years ago) #63

skidracer: You're right - adding a pointer to the xmlDocument seems to fix it. But there's something I don't understand - how is the garbage collector deleting something before the program is done with it? Isn't it supposed to check if there are any references to an object before deleting it? (and therefore removing the possibility of the garbace collector deleting an object before the program is done with it like what has happened here)?

Update: Version 2.19 will hopefully fix any problems related to debug/release inconsistancies.

skidracer

(Posted 4 years ago) #64

The list is being collected because the only thing left in use are the links, the list destructor needs to break the circular reference nature of the link set so the problem is the behavior of the list destructor which shouldn't affect you but it needs to force _head._value to null in order it break another circular reference which is what is causing the problem.

Mark Tiffany

(Posted 4 years ago) #65

Short of making _head private we're bashing our heads on a rational fix for this.

So the proper fix would be to have Private work inside Types?

Please? Can we please see Private fields / methods / functions in Types sometime soon? Pretty please? I've got some cherries handy...?

John J.

(Posted 3 years ago) #66

Update: MaXML 2.20 fixes a bug where manual ASCII escape codes (like "") were not being parsed correctly.

You can get the latest version here.

dooz	(Posted 3 years ago) #67

There is a bug in the xmlDocument.Create() method (or in BlitzMax, but I can't reproduce it). When I call it without an argument such as in:

Strict
Import pub.maxml
Local xml:xmlDocument = xmlDocument.create()
End

I get:

Building test3
Compiling:test3.bmx
flat assembler version 1.66
3 passes, 2283 bytes.
Linking:test3.debug.exe
C:/Documents and Settings/Paul/My Documents/Devel/blitzmax/maxml/.bmx/test3.bmx.gui.debug.win32.x86.o(code+0xc6): undefined reference to `_1'
Build Error: Failed to link C:/Documents and Settings/Paul/My Documents/Devel/blitzmax/maxml/test3.debug.exe
Process complete

The function is:

Function Create:xmlDocument(Url:Object = "")
Local doc:xmlDocument = New xmlDocument
If Url <> "" Then doc.Load(Url)
Return doc
End Function

By the way, you should probably have written not in terms of strings. Instead, using:

Function Create:xmlDocument(Url:Object = null)
Local doc:xmlDocument = New xmlDocument
If Url Then doc.Load(Url)
Return doc
End Function

Brucey

(Posted 3 years ago) #68

Try a non-Quick build of the app.

Sometimes, if you've changed a module, a quick build can still be trying to link to module code that no longer exists...

..just a thought.

dooz	(Posted 3 years ago) #69

I never use quick build, have had problems with it in the past. So it's not that.

Thanks anyway

pappavis

(Posted 3 years ago) #70

What are the chances of implementing some Xpath query possibilities?

I was thinking about a method like:
* SelectSingleNode(strXpathQuery) --> Returns a xmlNode

Both MaXML and the Dotnet Compact Framework 1.1 both doesnt have Xpath (I may be mistaken). I'd prefer to use a quick 'n simple XPath query to retrieve a node or attribute value, instead of using a iterated loop.

This is the XML for which i'd like to use;

<?xml version="1.0" encoding="utf-8"?>
<objects>
	<object id="1" type="type1" image="gfx/plaatje1.jpg" animframes="0"/>
	<object id="2" type="type2" image="gfx/plaatje2.jpg" animframes="0"/>
</objects>
<levels>
	<level level="1">
		<player itemnumber="1" initiateEnemyWhenPlayerY="120">
			<enemy uniqueID="1" type="type1">
				<movements id="1" startX="230" startY="291" destinationX="540" destinationY="475"/>
				<movements id="2" startX="540" startY="475" destinationX="610" destinationY="231"/>
				<movements id="3" startX="610" startY="231" destinationX="-120" destinationY="-200"/>
			</enemy>
			<enemy uniqueID="1" type="type2">
				<movements id="1" startX="610" startY="231" destinationX="-120" destinationY="-200"/>
			</enemy>
		</player>
		<player itemnumber="2" initiateEnemyWhenPlayerY="230">
			<enemy uniqueID="1" type="type2">
				<movements id="1" startX="230" startY="291" destinationX="540" destinationY="475"/>
				<movements id="2" startX="540" startY="475" destinationX="610" destinationY="231"/>
			</enemy>
		</player>
	</level>
</levels>

Overall, thanx for a great job :)

SebHoll

(Posted 3 years ago) #71

Hi John,

Brilliant module - it makes it quick and easy to save/load xml files. One problem though - if a file string is passed to the Load() method of an xmlDocument, and the file isn't an actual XML file, the entire program crashes out.

Would it be possible to gracefully return a value of False from the Load() method if an xmlDocument suspected to be FORMAT_XML is only a standard non-XML text file, instead of it trying to be unsucessfully parsed? At the moment, if a user accidentally opens a regular text file by mistake, I can't catch the error and the whole program terminates. :(

Cheers

Seb

John J.

(Posted 3 years ago) #72

Update: MaXML 2.21 fixes a bug where some parse errors are not handled correctly (crashing the application).

You can get the latest version here.

Seb: Thanks for letting me know about that, it's fixed now. Now all XML parse errors should be thrown properly, so you can catch them and respond to the situation appropriately.

pappavis: I'm not sure what an Xpatch query is, but from your description, it sounds like FindChild() / FindChildEx() / FindSibling() would work (though I'm probably wrong). I probably won't be adding many new features to MaXML any time soon (though I'll always try to provide quality support :) ), but maybe Brucey's libxml has the features you need.

SebHoll

(Posted 3 years ago) #73

Cheers John - brilliant job! Works perfectly and I can easily handle the errors using Try/Catch.

Brucey

(Posted 3 years ago) #74

John, an XPath looks like a directory path that points to a certain node in a tree.

<levels>
	<level level="1">
		<player itemnumber="1" initiateEnemyWhenPlayerY="120">
			<enemy uniqueID="1" type="type1">
			</enemy>
		</player>
	</level>
</levels>

The XPath to enemy might look something like : levels/level/player/enemy
(which actually refers to *all* enemy nodes for player)

But it's a lot more flexible, since you can also access paths based on ranges of attribute values and such like.

:-)

John J.

(Posted 3 years ago) #75

Update: Fixed a bug where &#xxx; codes were not being processed inside of quotes, and & codes were not being processed at all.

You can get the latest version here.

jsp	(Posted 3 years ago) #76

Thanks for update, did just some tests and works fine now with my xml attributes.

Filax

(Posted 3 years ago) #77

Many thanks for this great piece of code :)

Vertex

(Posted 3 years ago) #78

Dear John J.,

I use MaXML currently but here thinks that would be nice:
- No Pub Module Scope. Please use an own like JohnJ.MaXML
- Set it into SuperStrict mode. There are only the constants and some integer values to bring it into XYZ:Int form
- There is no need to import BRL.Retro
- A node without childs is saved like "<pause length="2"/>" it where nice when it saved to "<pause length="2" />"
- Node levels have actual 2 white spaces, 1 white space would be nice:

<song>
  <info>
    <interpret>Smash Mouth</interpret>
    <title>All Star</title>
  </info>
</song>

should be:

<song>
 <info>
  <interpret>Smash Mouth</interpret>
  <title>All Star</title>
 </info>
</song>

You can eventually define a method for userdefined whitespaces like tabulators, 3 white spaces or so on.
(But can you tell me now, how to edit _WriteNode to use 1 white space and solve this "/>"-problem?)

What is with encoding? Is that utf-8?

Thank you for this module!

cu olli

MichaelB

(Posted 3 years ago) #79

If you searched for "/>" within the module's source:

Function _WriteNode(File:TStream, Node:xmlNode)

...
	If Node.HasChildren() = False Then
		If Node.Value = "" Then
			File.WriteLine Indent + "<" + NodeContents + "/>" 
		Else
		    File.WriteLine Indent + "<" + NodeContents + ">" + _AddEscapeCodes(Node.Value) + "</" + Node.Name + ">"
		End If
...

I think you'll find the correct spot ;D

The module relies on strictly used utf-8... if your encoding has some mistakes in it, it will throw an exception (as long as my brain reminds me correctly).

bye
MB

Vertex

(Posted 3 years ago) #80

Yes, I have edited the wrong file... So there where no changes in the output xml files.

Also I have edited WriteNode_ to:

	...
	Indent = Indent[ .. Node.Level]
	Indent2 = Indent2[ .. (Node.Level+1)]
	...

Now it works, thank you!

I will test the encoding with german umlauts �, �, � or the german s-z-ligation �

cu olli

thalamus

(Posted 3 years ago) #81

I'm *really* keen to use this but the program seems to throw up an error when it encounters lines such as:

<!DOCTYPE DatabaseInventory SYSTEM "DatabaseInventory.dtd">

Removing this from the files isn't really an option - any chance of a fix? :)

Leon Drake

(Posted 2 years ago) #82

hmm whenever i use this command

Import pub.maxml

Local indexdoc:xmlDocument = xmlDocument.Create()

I get
Linking:eoled.exe
C:/Program Files/BlitzMax/endsoflegend/.bmx/eoled.bmx.gui.release.win32.x86.o: undefined reference to `_1'
Build Error: Failed to link C:/Program Files/BlitzMax/endsoflegend/eoled.exe

jsp	(Posted 2 years ago) #83

You could use New instead of Create

Import pub.maxml

Local indexdoc:xmlDocument = New xmlDocument