TMS Smart Setup: custom builds, git/subversion and versioning

Hi,

I've started using SmartSetup and would like to ask a few question about some of my findings/use cases. Maybe somebody can point me into the right direction or answer right away (I've read the documentation and blog posts and watched the videos, which already was very helpful)?

Here are my questions:

  • I am trying to build/package/install my own sources - is there some documentation about the tmsbuild.yaml file?
  • The tms.config.yaml has some references to git/subversion and the commandline has the repo-register, repo-unregister and repo-list commands. Can anybody explain how to use these (to possibly build my own sources from a git/subversion repo)?`
  • Working on my own packages, I often need to switch between versions. I've tried to follow the "alternative registry key" explanation in the Smart Setup beta youtube video, but do not see how to use this actually? Have two SmartSetup folders and use each for seperate tms.config.yaml and package versions? Can somebody maybe comment on the "best practice" for this or can show a sample setup?

Thanks!

  • I am trying to build/package/install my own sources - is there some documentation about the tmsbuild.yaml file?

Smartsetup was designed from the start to be able to build any packages (ours or from others), so this is a use case we care about. You can even build your exes with smartsetup, not only packages. But sadly our resources are limited, and we haven't have got time to document how to create your own smartsetup bundles. Still, you do have comments in the yaml file, and we can answer whatever questions you have, until we get time to write proper docs.

  • The tms.config.yaml has some references to git/subversion and the commandline has the repo-register, repo-unregister and repo-list commands. Can anybody explain how to use these (to possibly build my own sources from a git/subversion repo)?`

Again, this is in our todo list to document. We are looking at doing a blog post about it too, but we want to finish some other stuff first. But I'll give you an example with an actual product that you can try, so it should be easier to understand:

 tms repo-register GIT https://github.com/landgraf-dev/aws-sdk-delphi

This will register the aws-sdk repo from github. This repo must have tmsbuild.yaml at its root: aws-sdk-delphi/tmsbuild.yaml at master · landgraf-dev/aws-sdk-delphi · GitHub

tms repo-list

This will list the repositories you have registered.
And finally,

tms repo-unregister landgraf.aws.sdk

will unregister the repository we registered at the top. Note that you don't pass the url here, but the product id, which is defined in the tmsbuild.yaml, and it is the one reported by tms repo-list

If you register the aws repository (and don't unregister it), it will behave like a normal smartsetup product. You can now do:

tms install landgraf.aws.sdk

And it will install it. tms update will update it along with the other products. For most purposes, your product id "landgraf.aws.sdk" is now one of the products smartsetup can manage. You can install it, uninstall it, update it, use it as a dependency from other project, whatever.

Working on my own packages, I often need to switch between versions. I've tried to follow the "alternative registry key" explanation in the Smart Setup beta youtube video, but do not see how to use this actually? Have two SmartSetup folders and use each for seperate tms.config.yaml and package versions? Can somebody maybe comment on the "best practice" for this or can show a sample setup?

Yes, the idea is that you have multiple smartsetup folders, each one with a different registry key. Then you start bds.exe /r registrykey and can use one or the other configuration.

We have an internal tool that we also plan to give away that allows you to select an ide with the registry key, and start that IDE version. It also allows you to automatically update the smartsetup for each IDE, so you can have completely separated development environments without VMs and with just one Delphi installed. That's one of the reasons we want people to be able to move their own components to smartsetup: If you have them all in smartsetup, you can easily update the different registry configurations.

Here is a small gif of how the tool is working today (not 100% sure the give will show, and if it will repeat. If you can't see it you can click here):

As you can see, we launch 2 delphis with different registry keys: one for development and one for testing. One is in light mode and the other is in dark mode (so you can easily know which one you are working on) and they have a completely different set of installed components (each set comes from a different smartsetup folder with a different registry key)

Once again, it is all a matter of time. There is so much we still have to do with this, and so little time...

I just remembered that we don't ship the "template" tmsbuild.yaml with the products, so I am attaching it here.

tmsbuild.yaml.zip (2.6 KB)

Hopefully it can give a clearer picture on how to use it. It isn't really difficult, but the things you need to know are:

  • The packages must be inside folders with the some of the names: 'lazarus', 'd6', 'd7', 'd2005', 'd2006', 'd2007', 'd2009', 'd2010', 'dxe', 'dxe2', 'dxe3', 'dxe4', 'dxe5',
    'dxe6', 'dxe7', 'dxe8', 'dseattle', 'dberlin', 'dtokyo', 'drio', 'dsydney', 'd11', 'd12'
  • tmsbuild.yaml should be at the root of your distribution.
  • We have a "framework" concept in tmsbuild.yaml. Frameworks are just definitions of the IDEs a package support, and the platforms (win32, win64, etc). Then, for each package, you need to use at least one of the frameworks you defined. You also have the special names "design" and "runtime" for design packages and runtime packages.

As said, any doubt you have, we will be happy to answer it.

Hi Adrian,

thank you very much for the detailed explanation and the commented tmsbuild.yaml - that's exactly what I needed (and more, fantastic!).

Another thought/question: Did you also think about CI/CD? What could be the typical use for setting up components and running builds on a build/CI machine? Fill the Products folder from e.g. a git repo (so we have a defined set of component versions)? Or do you as described for the 3rd party packages: pull sources from git and build/install?

If doing the later (tms repo register), how can we specify which branch is pulled during tms install?

Thanks and keep up the nice work!

We internally are just filling the product folder with the version, and then calling tms build on it. The tms repo-register is more so you can publish your own components and have them integrate with smartsetup, but it could be used for CI too, we just haven't looked at using it that way.

One of the many ideas for the future is to have a centralized github repo with only tmsbuild.yaml that is automatically "registered" in smartsetup (without the need to do a tms repo-register) and the rest of the repo would be wherever it is. This would allow users to create yamls for any open source delphi component, without having to ask the open-source maintainer to add a tmsbuild.yaml to the component's repository. (but for this to be really useful, we need to relax the naming conventions so we can use whatever the open-source repository uses, that is also in our todo-list).

About versioning and branchs when doing a tms repo register: As you can guess, this is also "work to do", but it falls into a bigger topic which is supporting old versions. We have pages and pages of discussions about the best way to implement them, but it is not really decided yet. It looks like it will be something like:

tms install tms.flexcel.vcl:7.10

To install that particular version. For a repo registered with tms repo-register, the part after the ":" would be a "tag" for the repo, for example, any of the tags here:

Or might be the releases instead, this isn't decided.

Finally, about the branch. We don't plan into supporting them directly, but our git support is very simple, it just calls git.exe (no built-in library to call git), and you can specify the git parameters in tms.config.yaml:

  git:
    # Location where to find git.exe. If not specified, smart setup will search for the first git.exe available in the OS PATH.
    # git location: c:\git\git.exe

    # Here you can specify extra parameters for the git clone call. If not specified, smart setup will do a plain clone.
    # clone command: clone

    # Command for git pull
    # pull command: pull

    # Command for a shallow clone that will be used to get a single file.
    # shallow clone command: clone --depth 1 --filter=blob:none --sparse

  svn:
    # Location where to find svn.exe. If not specified, smart setup will search for the first git.exe available in the OS PATH.
    # svn location: C:\fpc\fpcbootstrap\svn\bin\svn.exe

    # Here you can specify extra parameters for the svn checkout call. If not specified, smart setup will do a plain checkout.
    # checkout command: checkout

    # Command for svn update
    # update command: update

    # Command for svn export
    # export command: export

By uncommenting the "clone command" line, you can change it to be clone -b <branch>and also the shallow clone command.