JADE Environment Development Ideas

We are planning to run a Source Management workshop in conjunction with our TOI in June... hope you can make it and share your ideas and workflows! Either way thanks for this useful feedback.

Best,

Alenti

We are planning work on the source control, this idea represents multiple ideas and considerations that we need to investigate further.

Sounds really good - merging of snapshots would be valuable and a huge timesaver for our team.

Link to the JADE IDE Extensions idea may be handy :)

These latest comments highlight the need to maintain a clear line of separation between the IDE and source control.

At the initial 'source control summit' meeting some of us attended (circa 2016?), JADE discussed how they were apprehensive of plugging the IDE into any particular source control system due to our various needs & preferences.  By the end of this, we left with an understanding that JADE would provide platform improvements to support those implementing source control using any of the various options available (i.e. friendly extract format without user or environmental details).  However, further consultation didn't transpire and it would seem the approach currently favoured by JADE is to plug Git into the IDE directly.

I'm not saying JADE shouldn't implement a source control plug-in (they need a system that works for them too!!), but the approach taken needs to be a strategic one as some of us already have well established processes using other systems (potentially built in-house).  While any source control solution provided by JADE will be useful to those who don't already have such processes in place, those who do are unlikely to migrate unless it can replace or integrate with issue tracking, change management & automated release/deployment systems already in use.

If JADE were to prioritise implementing IDE extensions, this would not only become the vehicle for JADE & the wider community to implement multiple source control plug-ins, there'd be added value due to the platform improvements needed along with the general ability to implement extensions for various purposes (integrating with issue tracking & change management being one).

We are planning to run a Source Management workshop in conjunction with our TOI in June... hope you can make it and share your ideas and workflows!  Either way thanks for this useful feedback.  

@Gaurav with re your 3rd bullet point (Data support)
Running with several versions of the same classes and objects in the same environment might be quite an undertaking? That would be my only reservation for it being on the list of minimum viable requirements.

Furthermore there are some existing strategies and some new ones that would come from having the above basics in place that might reduce the need for this:
- utilising generated data rather than copies of live dbs.
- consideration for using more than one development environment
- versioning of ‘breaking’ db changes (eg. property removes, class moves etc), that all other dev branches are then pegged against.
- use of migration scripts (both forward and back)
- restoring from backup and ‘pull’ your code changes back in again

To extend on some specifics regarding what functionality I would like to see for effective source control integration:

• For integration with version control such as Git, changes made by each developer needs to be recorded at the most granular level as possible. This includes all things such as compiling changes on a method, structural changes such as adding/deleting/renaming entities on any level (Class/Schema/Interface/RelationalView)
• This level of granularity will be largely different from current functionality, with schema extracts. As such, a lot of associative information such as class constants, global constants, locale information, and translatable strings will be lost and needs to be handled separately. Possible ways this can be done is by maintaining a backend database that constantly holds the structural integrity of the current source code, or having dedicated files that store this information.
• Currently, we have our primary development environment setup with Data. When structural changes are made such as deleting a reference, the data for that reference for all instances will be deleted/purged. Scenarios like this we cannot selectively repair by undoing changes (adding back the reference). This is a concern when developers are working on different features residing on separate branches. If in one branch the "reference" is not available or deleted, the persistent data for that Jade environment would be deleted. However, when we switch over to the other branch where the reference is present, the user/developer expects the instances to still contain the data associated with that "reference". Simply put, maintenance of persistent data might need to be given special consideration while making changes granular or atomic in nature.
• A reliable log of all changes/commits that are made. However, rather than containing snapshots of the current schema structure or the full method source, these changes should have a similar approach to version control such as Git, so as to record the source code lines added/deleted relative to a base/master branch. Similarly, it would need to record additions and deletion of attributes/references relative to a base snapshot, rather than making a whole new snapshot of the current schema. Furthermore, they would, of course, maintain some basic info such as the user/developer who made these changes.
• Being able to encapsulate these granular changes into commits, which would have the bare minimum information such as time, date, developer, GUID and a small description. These commits would then be easily able to be reverted, transferred to a new branch, merged or split into commits, existing features of established version control systems. 
• Git/SVN/TFS integration - The splitting up of these changes and features needs to be fully compatible with these existing systems. The specific format in which how hierarchy is stored, data-types and source content are maintained is still up in the air and open for discussion. But once fully integrated with a proper structure, these granular changes should be seamless with a versioning system. If a corresponding database or file is updated with every granular change, this would mean that every time a method is compiled, a basic command/structure change has been done (such as adding an attribute, defining a constant, deleting a subclass) : Immediately a file needs to be generated to indicate the delta/change that happened. There should effectively be no reason a user needs to extract or load data for simply committing changes. When pushing or finalizing changes, more complex operations may be left such as re-organizing data structures and persistent data formally for it.
• Code Diffs! The more atomic changes are recorded, the easier it becomes to generate these. Functionality should be extended to diff between any two unique commits/GUIDs such that we can retrieve all the operations that occurred in-between, which includes not just source changes, but also structural changes, perhaps a data type change for a certain RPS field in the RelationalView

Of course, since maintaining the Jade environment is not the same as a general IDE, we need to consider all aspects such as variation in code, change in data structure, change in SQL mappings through RPS, Form changes and possibly further such as jade environment variables and information associated with 'singleton' like entities like system, app, global, node and process whenever applicable.

Having said all that, these are still simply starting thoughts to integrate Jade with effective version control. Coming from using GIt in the past, I can't help but hope the possibilities and efficiency that can be achieved by implementing this properly!