GSoC Status Report 8/13-8/19

A few days late with this status report, quite overwhelmed by starting my new semester of classes. Things have been going well enough but I was unable to wholly integrate the TPCL EE into the RDE. Rather than rush to complete an inferior product I decided to postpone completion of the project until after things got settled here. Ah well...off to class >.<

-Fro

GSoC Status Report: 8/6-6/15

This past week went pretty well. Things moved pretty quickly with the Expression Editor. Features added include:

• Indentation in Expressions
• A new GUI layout somewhat reflecting mithro's comments and input
• Added support for variable length expressions...that is...expressions that can have an arbitrary number of subexpressions [(+ arg1 arg2 ...), (cond ((test1) (when-true)) ((test2) (when-true))), etc]
• Literal expressions supported as well (numbers, strings, etc)
• Refactored the internal representation of Expressions
• Put in the BlockStore which maintains the TpclBlocks in a nice structure and will allow blocks to be referenced by an id number so that we can have some persistence and also establish an MVC pattern with things that have to display the tree.
• Also added a "Quick Insert..." menu that allows access to commonly used functions and symbols, I plan on making this editable by the user.

For this coming week (the last week of GSoC) I plan on making all of the current features of the TPCL EE work consistently and as bug free as possible and then integrate it into the RDE. Some convenience functionality may not make it in by the end of the program but I'll add it in slowly but surely on my own time.

-Fro

On The Generalization of Servers

At the end of last week nash asked me to give some thought as to how much of a ruleset the RDE could be used to develop. At the moment there's not that much that it can be used for, aside from things like components, properties, designs, resources, or planetary systems and the like. And entire ruleset cannot be constructed from within the RDE. That does not mean that there cannot be, only that at the present moment it is not possible. I have given some thought to the issue, however, and believe that it would be possible to design the majority of a ruleset in a server-independent form if a few things were to occur:

1) A basic set of Order types would be available for use by a ruleset from a server. That is, a ruleset could choose to make use of a stock set of common Orders such as move orders, fleet construction orders, colonization orders and the like. It would also be possible for the ruleset to not use the available orders, or to simply extend and override the base Order with its own.

2) Any new major features (such as resources and trading as seen in TimTrader) should be implemented in a server-agnostic format. Instead of writing the plumbing in Python, as is the case right now, the TPCL language should be extended and exploited. In much the same fashion as Components affect the value of a Property so too could TPCL (or rather, TPSL [Thousand Parsec Scripting Language]) code could be written that allows Producers to generate Resources on a planet and the Consumers to consume those same resources. With a bit of forethought, then, it would be a fairly simple process to put the majority of the processing and functionality of the feature in a server-neutral format which could be stored, edited and tested in the RDE.

Now, certainly not all Orders can apply to all rulesets. And different rulesets will have different variations on the same Order. And it would not be the most efficient thing to do to go about interpreting TPSL code for all of the interesting things that happen on the server. However I believe it should be a goal of the TP project to make the building of new rulesets accessible and easy.

Games like Neverwinter Nights have such a large and active fan and hobbiest base because of the fact that it is quick and easy to create objects or features and share them. More importantly it is extremely easy to use the work that other people have already done in your own efforts. The more Orders that the servers can provide for use by rulesets by default the quicker rulesets can get up and running. If the ruleset needs functionality that isn't already present then it can be put in manually, hopefully to the benefit of all the other rulesets as well.

The same applies to features. Making resources, for example, accessible and easily sharable would make it more likely for people to make an effort at starting a creation of their own.

Anyway, those are my two cents (or rather, two ideas). I'm sure there's plenty that I haven't considered, but it seems to me that there a bit of an opportunity being missed in some ways.

-Fro

TPCL Expression Editor Taking Shape

Well, as can be seen above the TPCL Expression Editor is taking shape. After taking a look at the R5RS specification I decided that going about trying to fit TPCL expressions into categories would be artificially limiting and error prone. The result is that I plan on displaying all available tpcl blocks, both built-in and custom templates, in a tree control. These will be inserted by clicking on an expansion point and choosing the "Insert" option. I will also likely add a small menu within the insert menu that will expand into a subset of the available built-in expressions so that booleans, literals, arithmetic functions and other similar simple expressions can be quickly and easily accessed (thanks, mithro, for that idea)

Anyway...things are progressing well. Expressions created with the editor are now formatted nicely with newlines and indentation so that the output is still human readable. Anyway. More work tomorrow.

-Fro

GSoC Status Report: 7/30-8/5

Meh, quick status report. At this point it's still probing around in the dark a lot so there's not a concise checklist that I can mark off on.

I got a basic build of the expression editor working last week which made me happy. I've also got some thoughts put together for nash about how the servers can be massaged to make them more amenable to rulesets being written in a form that is general enough to be applied to both with minimal effort. A post with my thoughts in that respect will be coming later today.

Anyway, back to coding. Working in a new view for the TPCL Blocks and making the code nicely indented. Screenies later.

-Fro

A few more thoughts on the TPCL EE

What I've got in place works just fine for predefined expressions. However there are going to be issues with expressions that take a variable number of arguments (i.e. (- 1 2 3 4 5...)). That and allowing the user to put in numbers and string literals. But that's an issue that will be dealt with as I develop the editor itself. Now that I have something concrete backing up what I'm doing I'll be able to better find solutions to these issues.

Functional TPCL Editor

So. I got the GUI working. All of the stuff in the screenshot was put in with mouseclicks. The progression was:

(lambda (design) *EXPR*)
-> (lambda (design) (if *BOOL* *EXPR* *EXPR*))
--> (lambda (design) (if (= *VAL* *VAL*) *EXPR* *EXPR*))
---> -> (lambda (design) (if (= *VAL* *VAL*) #t *EXPR*))

And I got the TpclExpression class to the point where it actually functions pretty well. Expressions can be inserted and removed from other expressions which is basically what I've been aiming for. This will remove the human error of hand coding and make a large database of premade code snippets available (at least, that's the hope).

So now that I know the system works I just need to make sure that it's basically what mithro, nash and llnz wanted out of this and then I can start moving forward with introducing nice formatting to the expressions and expanding on the database of TPCL code.

I had originally planned to attempt to limit the types of expressions that could be nested inside of others. But that's just silly. It will introduce more problems than it's worth, I think. At most pointers can be added that help a ruleset designer make smart choices.

-Fro

TPCL Code Manipulation

Let's take a trivial piece of scheme code:

(lambda (design) (= #t #t))

Doesn't do much, does it? Don't need an editor to help you develop it? No. Not really. But it's a start. The following python code generated the above expression...and it should go without saying that this is really intended to be driven by a GUI and not manipulated by hand...but still...cool, eh?

>>> from tpcl.data import Import
>>> blocks = Import.LoadBlocks('tpcl/data/base_blocks.xml')
>>> exp = Representation.TpclExpression(blocks["INITIAL_BLOCK"]["Lambda Design"])
>>> bool = Representation.TpclExpression(blocks["BOOL"]["="])
>>> t1 = Representation.TpclExpression(blocks["VAL"]["#t"])
>>> t2 = Representation.TpclExpression(blocks["VAL"]["#t"])
>>> exp.InsertExpression(17, bool)
>>> Setting offset of expression to 17
>>> bool.InsertExpression(20, t1)
>>> Setting offset of expression to 20
>>> bool.InsertExpression(24, t2)
>>> Setting offset of expression to 23
>>> str(exp)
'(lambda (design) (= #t #t))'

-Fro

Expression Editor

I went ahead and threw together a possible layout for the TPCL Expression Editor in wxGlade today. It was based heavily off of the mockups I did before my project was chosen. A screenshot is shown below:


















I'm also working on a way to make the expression editor useful for those who don't really program. That is to say, it will simply be a matter of pointing, clicking and selecting what you need. And if that's all you do there should be no errors in your code. Code snippets should be available in plenty and pre-constructed blocks of code should provide common functionality quickly and easily. Lofty goals, eh?

Right now I feel like the way I'm going about it is too brute force and inelegant, but I'll get this working in a basic prototype form and if I still feel issues I'll go back to the drawing board and do some more research.

-Fro

GSoC Status Report: 7/23/07 - 7/29/07

Not so much to report on this one. No list of accomplishments. I spend the first part of the week failing to get a context menu working for the tree view. The way wxPython sends its mouse events defies my ability to grasp. I can't prevent the selectionchanging and selectionchanged events from firing before the right click event and so can't veto it. Feh. That frustrated me to no end. So I'll get back to that later from a different angle after I get the TPCL Expression Editor working. Oy. I mean. I even messed with where I bind the handlers and didn't get the results that I wanted. Meh.

I also went ahead and provided support for multiple categories now that tpserver-cpp supports them as well. They're included in the RDE and in the generated code. So that's kind of cool.

The end of the week was spent working on figuring out how to get the StyledTextCtrl to recognize keywords for the TPCL Expression Editor without me having to handle styling events with a lexer of my own. Thankfully I found something that should work.

Now I'm working on getting the TPCL EE up and running. It looks to be a bit bigger of a project than I had imagined before, but I'm confident that I'll make good progress this week.

-Fro

Oh. And since mithro is always prodding me to post new and fabulous screenshots of the RDE, here's a shot of the feedback the RDE gives on errors:

Oy, the commits

I am quite terrible about checking my commits. I'll charge on in and commit something that breaks everything else, simply because I forgot to add a new file. It's getting rather silly now. Every time I create new files I forget to add them to git. Meh. And then I make posts about how awesomely everything works. And it doesn't work when it gets checked out. Feh.

-Fro

GSoC Status Report: 7/9/07 - 7/22/07

This status report will cover two weeks instead of one since I missed the last report.
Things that were accomplished this period are:

1. Switching to the Elementtree library for XML input and output
2. Updated the XML persistence file format
3. Added a global configuration manager which provides a means of tracking recent projects and other handy info like that
4. Ability to rename objects
5. Added project validation - All attributes must be filled in and TPCL code must be valid. Syntax is checked for the TPCL code and also all procedures and symbols used are verified as being valid.
6. Changed how we monitor changes entered in the edit windows - no more funky delayed behaviour
7. Added the ability to duplicate objects - handy for making a bunch of components that do pretty much the same thing to varying degrees.

On the plate for this coming week is:

1. The TPCL Expression Editor - fun, fun
2. Support for more types of game objects.

-Fro

RDE Fully Functional

I'm sure that there are bugs just waiting to be discovered with more rigorous use. And the fact that game objects only support one category at the moment (which is also the case with tpserver-cpp) keeps things from being perfect, the RDE is at a state that I'm quite happy with now. I just went through a quick run of duplicating the Minisec objects from tpserver-cpp in the RDE from scratch. I found a few small bugs along the way, but they were easily fixed. The generated code worked without a hitch.

TPCL syntax checking and code validation is in place so that people don't have to deal with cryptic error messages from the servers when they try to run their code. The RDE even checks to make sure that the project doesn't have any errors before generating code.

I'm can now turn my attention towards developing the TPCL expression editor. That will be quite fun, I think, and also quite challenging, but at least a lot of the work is already in place with the StyledTextCtrl that I'm using to display the TPCL code. It already provides facilities for code completion and the like so extending it to incorporate what will basically be code-snippits-on-steroids shouldn't be too bad.

Woot.

-Fro

GSoC Status Report 7/2-7/8

This week was pretty uneventful. The Fourth of July ended up taking a lot of time, both before and after, so it turned out to be a bit of a holiday week. The things that I accomplished were:

1. Fixed deletion behavior. Objects that are deleted now propagate the event to all other objects so that they can remove associations to it.
2. Changed the method for loading edit panels, a new wxPanel isn't generated every new selection, a pre-existing one is simply filled with new data (thanks mithro)
3. Refactored the code to separate GUI stuff from the guts. Also separated the various representations of game objects a bit. More work needs to be done on the XML view, but it's better than it was. This was fairly easy to do since there was already a lot of separation and decoupling between different elements of the program.
4. Started work on adding icons to the tree control - this allows immediate feedback for modification events and allows more expressive visual elements to be incorporated into the RDE.

In the coming week I hope to produce something that someone can simply get started running and see immediate benefits from doing so. This means that I want to finish fleshing out the UI and adding functionality. This includes:

1. Adding support for other object types, perhaps resources, solar systems and planets.
2. Doing basic error checking on game objects to ensure that all fields are filled in before allowing code to be generated, I may also incorporate Scheme syntax checking as well if it isn't all too difficult.
3. Allowing for the renaming of objects (this will likely involve extracting the object save routine from the individual objects and making it a general method)
4. Finally talking to llnz to get his ideas on the generated code for tpserver-cpp
5. Getting back to mithro about the interactions between the RDE and tpserver-py

-Fro

GSoC Status Report: 6/25-7/01

This week went by pretty fast but I was able to get a lot accomplished. Things that were completed include:

1. New save/load scheme for objects - persistence files are only created after an object is saved, not when the object is created in the RDE.
2. Made saving objects work correctly, which included implementing primitive modification detection for objects. This will be made more elegant in the future as right now it's a rather simplistic and ugly way of doing things.
3. Made all of the object editing panels load from XRC files to make it easier to get more complex layouts of widgets and modify the layouts later.
4. Made it possible to view and edit the TPCL Cost functions associated with the Properties that Components affect.
5. I imported the Minisec Components and Properties and made it its own project to test code generation with.
6. Got code generation working for tpserver-cpp. This involved getting tpserver-cpp to compile in the first place, which was somewhat frustrating. But once that was done code generation went along without all too much of a hitch, just a few typos that were easily fixed.
7. Added the Category object to the RDE and gave it code generation support. This is incomplete at the moment and will require editing of tpserver-cpp to provide support for multiple categories on an object.

My plans for the following week are as follows:

1. Fix performance issues with panels being loaded. Right now processor usage spikes to insane levels. I'm pretty sure this is due to STC's being generated for every panel at every change in object selection, so it won't be too hard to fix.
2. Adjust code generation so that the code that it can be used by making only one function call.
3. Work on supporting multiple categories in both the RDE and tpserver-cpp
4. Whatever else comes up throughout the week. I'm quite satisfied with the progress that I made at this point, with the midterm approaching in one week's time. I'll be looking just to polish things up before moving on to working on a TPCL expression editor.

-Fro

Fro's tpserver-cpp

My testing server can be accessed at jgard.dyndns.org, just point your TP client that way. There are no guarantees that it will be up and running, but when it is it will always allow new players to join.

-Fro

To the left we see the precompiled version of tpclient-pywx connected to the modified tpserver-cpp that I have running. The server is running using code that was generated by the RDE. The FroHull component was generated completely from within the RDE with no C++ code written by my own self to get it working aside from a simple call to an initComponents function. A slight modification to the DesignStore in tpserver-cpp allows Components to be referenced by name and makes code generation for these basic elements quite easy.

In the process of getting the generated code to work within the current Minisec ruleset I discovered that to adequately assist the development of rulesets the RDE will need to edit and track a number of other "objects" from within the game, the bare minimum being the design Category attribute.

Another thing that came up was the fact that there is no checking of the TPCL code being done, so when I first generated the code I had a tpcl cost function with a # where a number should have been, which resulted in a (to me) cryptic error during loading the game. I eventually want to validate components and properties, at least on a basic level, in the RDE before code can be generated.

I will probably have to work out a specific coding style to be followed for those wishing to use the RDE to generate their code, but I expected that from the beginning. Right now I'm just happy that the code that I generate works and can be used to generate properties and components by adding only four lines of code. The structure of the generated code will likely change with time, perhaps to make a single function call that will initialize all RDE generated objects so that it becomes trivial to use.

-Fro

GSoC Status Report 618-6/24

Two more features need to be implemented to make the RDE usable as an environment to store and edit game Properties and Components (editing TPCL Cost functions and renaming objects). That should be completed early this week and then a few more tweaks to the GUI (loading from XRC) will be the last that I want to do before I dive into code generation for tpserver-cpp.

Things that I accomplished this past week are:

1. Added a menubar and statusbar
2. Ability to add and delete objects
3. Ability to create projects
4. Ability to open projects
5. Added Scheme syntax highlighting for TPCL code
6. Fixed the sentinel properties (which I'll probably get rid of anyway since explicit is better than implicit)
7. Messed around with XRC to get some good layouts for editing objects
8. Ability to associate Properties with Components

Things that I want to accomplish this coming week are:

1. Conversion to XRC for most of the GUI (need to talk to mithro about that, feh, timezones)
   
2. Ability to rename objects
3. Ability to edit TPCL Cost functions
4. Maybe context menus for some useful functionality
5. Basic framework for code generation

As a peek at what I want to do with the GUI to make it more intuitive, below is the component editing panel as it currently sits in wxGlade. The big change is that associated Properties are listed in a listbox and their TPCL Cost functions are displayed and editable to the right - less clicking and easier to inspect this way.


-Fro

The RDE is coming along well. Syntax highlighting came easily thanks to the StyledTextCtrl which is powered by the Scintila plugin. Code folding and highlighting are all included. It'll mostly be just a matter of getting a proper keyword list for the subset of Scheme that TPCL uses. The code for implementing the syntax highlighting and code folding was all readily available in the wxPython demo application and was easily adapted to the needs of the RDE.

The RDE now has the ability to create and open projects, but not yet the ability to delete projects. Almost all of the features are in place for a ruleset designing to input the components and properties of a ruleset into the app. Once the internals of the app are worked out I can get to work getting code generation to work, starting with the C++ server (since mithro has so kindly offered to do code generation for the python server, woot)

I also poked around with XRC...and I have to say, mithro was right that it's pretty awesome. In about five minutes I was able to duplicate the current layout of the ComponentPanel with far less frustration than coding by hand.











-Fro

Been a while since the last update, eh? Well, along with this post comes a screenshot of the current state of the RDE. No menu bar or status bar yet, but there is some nice highlighting going one which shows a Component selected for editing and each of the Properties that it affects highlighted in red.

As for the current state of the project, read below:


What has been achieved so far:

1. Storage for multiple gametypes which is completely decoupled from the program itself
2. Dynamic loading of game object information to ensure that the only bloat is Python's inherent size
3. Game object data model and interface for listeners for changes to that data (which may not seem like a lot but was quite interesting to get working as I learned a good bit about properties in Python and initializing methods on the fly)

What is yet to be achieved:

1. Implement the interface that allows the program access to and the ability to modify game objects (adding new ones, modifying existing ones, etc, and ensuring that changes are propagated to all important places). Examples of features that will use this are highlighting made to objects that have been changed since the last update and highlighting properties affected by a given component.
2. Use a more robust and reliable method for reading in and writing out XML
3. Ensure that all global variables are properly set and include a setup script to ensure that directories are properly recognized by the RDE (as it is now it's dependent on the working directory which must by .../tpruledev/src/ which is just sloppy)
4. Code generation
5. Possible use of XRC and AUI for GUI as per Mithro's recommendations (poking around the development branch on tpclient-pywx for that)

Anyway, more later.

-Fro