Reducing Library Paths

[Rasmussen_Lance]

In the non smart setup installers, I could install into a custom directory where I could minimize the path names in order to conserve on the library path limitations, which can be troublesome with a lot of 3rd party components, like I use, and when compiling via command line.

I created a C:\TMS for smart setup, however, when I install all the TMS components that Smart Setup support, the paths become very long, which when factoring all the components, pushes the limitation.

I.E. C:\TMS\Products\tms.fnc.dashboardpack\packages\d12\Win32\Release (64 chars)

Yes, I could reduce C:\TMS to C:\T\ but that only gains a couple chars for each of the component sets.

Previously, the above would be on my machine, C:\TMS\FDPK\packages\d12\Win32\Release (38 chars).

Is there a way to set up the YAML config or add built-in environmental variable support, such Flexcel does, for each component, so that, using my example:
$(TMSFDPK) = C:\TMS\Products\tms.fnc.dashboardpack\packages\d12 so that the Library path could be $(TMSFPK)\Win32\Release (23 chars) or $(TMSFDPKD12) = C:\TMS\Products\tms.fnc.dashboardpack\packages\d12 for $(TMSFPKD12)\Win32\Release (26 chars) in the case of needing to be able to have environmental variable support for machines with multiple Delphi versions.

This would make things very efficient for the Library paths.

While I understand perspective of a few TMS components using Smart Setup may not seem a big deal, some of us like to install EVERYTHING in order to explore and work with everything. And also some of us do not have just a couple components.

Thanks in advance for any consideration or suggestions.

[adrian]

Hi,
Thanks for reminding us of this. It is something indeed being considered.

While I understand perspective of a few TMS components using Smart Setup may not seem a big deal

It is a big deal. One of the main reasons we started smart setup was to make it possible to install all of our components in a simple way, and minimizing stuff like the bigger library path or Windows path. We are already adding a simple entry to the Windows path for all components, which is the bigger problem, but we should also improve in the Library path.

Is there a way to set up the YAML config or add built-in environmental variable support, such Flexcel does, for each component, so that, using my example:

Not right now, but I am also not sure this is the definitive solution. It is indeed the simplest to implement for us, but:

1. In my experience with years of the FlexCel installer, the environment variables bring a lot of pain. Even if it is fixed now, for decades, Delphi would consider those variables as paths that don't exist, and delete them if you press the delete invalid paths button. There is a section in FlexCel docs to tell you about it: FlexCel Installation Guide | FlexCel Studio for VCL and FireMonkey documentation (See the screenshot and the red remark at the end).
2. They make it difficult to find out where the components are really installed. With the full path spelled out, you can just copy the path and paste it in explorer. With the variables, you need to copy the path, then go to environment variables and search for the variables to find out where they really are.
3. While they indeed reduce the library path, it is still huge. Just a little smaller so it fits in the 32k msbuild limitation, but not ideal either.
4. They just reduce it for msbuild. For dcc32 (what msbuild calls), the expanded paths are passed, so the problem is still there. It is true that the msbuild command line is a little longer than the dcc32 one, so using environment vars fixes some cases, in general the problem is still there.

So we are looking at other alternatives too. One could be to hard-link all dcus into a single "c:\tms.tmssetup\dcu" folder or similar, so you just add that folder to the library path and don't have to worry even about using short names as "c:\tms" or "c:\t" to shorten anything. A single entry will be added to the library path for all tms products. (that's what we currently do with the bpls and the Windows path). Another could be to change the output for all components to a single folder instead of hard linking them. And of course, there are a lot of variations that can be used: we could use a single c:\tms.tmssetup\dcu folder or we could use a folder per product line, like all fnc in a single dcu folder, all biz in another, etc. This might help with not having a too huge dcu folder. But well, it all has to be studied.

About workarounds/mitigations right now until we find out the best solution:

1. Smart setup itself will never run into this limitation, as it doesn't use your library path anyway. So even if you pass the 32k limit, smart setup will still work correctly.
2. When compiling from the command line, you can use /p:DCC_UseMSBuildExternally=true See Command-line for DCC is too long while using MSBuild to compile Delphi project - Stack Overflow
3. We can't of course fix installers from other component vendors, but they can fix it too by using DCC_UseMSBuildExternally. Maybe you can let them know in the ones you find the errors. Or they can move to smart setup once we make it generic enough to be used by everyone
4. You can have multiple registry entries to play with the components, so you don't have such huge search path in any of them. Having multiple delphi installs in the ide using /r used to be cumbersome, but you know, with smart setup and the option alternate registry key in the YAML it gets quite simple (at least for the components using smart setup...). I am using that myself, and also even not registering stuff at all (not adding it to the library path, or adding it to anywhere), and compiling with smart setup instead (which doesn't use/need the library path). That's a topic for another blog post, when I get the time. But there are many "undocumented" ways to use smart setup that make it simpler to deal with different installations/configurations and that we already use ourselves internally a lot. You can even have say product1 using FlexCel 6.22 and product2 using FlexCel 7.15 in the same machine at the same time (please always use the latest flexcel... but you get the idea). And have a CI server where you don't install anything at all (just build the dcus), and still can build your products. You can isolate different environments/components per product, and that is another world in itself. But again, that's a topic for a new blog...
5. Smart setup has a command "tms doctor", which will also help you keep your library path tidy. It will search for non existing entries and allow you to remove them (among a lot of other things). That might help too with keeping the library path in shape. But well, that's another cool toy that right now is only in our github repo, we haven't made tms doctor public yet. (and won't until we are 100% sure if doesn't break any stuff). But in the future it might be another tool to help here. As said, it is not that we aren't aware of this problem or we don't consider it important, it is just that there is so much to do, and so little time... but having a better library path is for sure in the plans.

[Rasmussen_Lance]

Thank you Adrian,

I'm glad to "bump" the need back into thought and greatly appreciate the insight you provided as well as some things for me to try. As you say, it IS a major thing, for such a minor thing.

The TMS DOCTOR sounds great. I have a small util I run that sorts the library paths, to help deduplicate the paths manually if needed. So this sounds good.

I appreciate the DCC_UseMSBuildExternally tip. That was new to me. That should help quite a bit.

I think you are on the right track for a potential solution to hard link into a c:\tms.tmssetup\dcu{platform}. Assuming there is not DCU conflicts with unit naming across all TMS products, this would work very well.

I use DevExpress VCL (subscription) and LMD (full) and while they have a LOT of component packages and units all together, they compile to a single DCU directory for each platform. So they've never been an issue.

It's just different with TMS due to the large number of products offered and trying to make it easy. If Bruno wasn't so helpful in having TMS serving the needs of the Delphi community with a LOT of useful and varied products, it wouldn't be a problem. [LOL]

Lance