Automating Imports

If you work with more than a few projects that import from a common project, you may want to automate importing from that project. Automating imports can be as simple as scheduling this command:

madbuild.exe -project "c:\example\example.flprj" -target "build-for-import.fltar"

 How does that work? Isn’t madbuild.exe for generating output content?

Yes. But some other things happen when a target is built.

Flare targets have an Auto-Sync feature that enables running an import prior to generating content. Auto-Sync can be disabled for a target from the Target Editor. The setting is on the General tab, under Auto-Sync: Disable auto-sync of all import files. When not checked, the option is specified in the target file markup as a DisableAutoSync=”false” attribute and value on the CatapultTarget element. The value is true when the box is checked.

Flare imports have a setting on the Source Project tab called Auto-reimport before “Generate Output”.  Targets with Auto-Sync enabled will prompt an import specified by the import files with the Auto-reimport before “Generate Output” option selected. The markup for this option is AutoSync=”true” on the CatapultProjectImport element.

The command line executable that enables starting builds outside of the Flare user interface is called madbuild.exe. It is located in the folder where Flare is installed. When a target is built with madbuild.exe, Auto-Sync is still run if specified. However if the project is under source control, madbuild.exe does not prompt for a checkout as happens in the Flare interface. But more about that later.

By the way, this post is a follow-up to a previous discussion about batch files called Batch Files, madbuild.exe, and commands.

Setting up automated imports is accomplished by scheduling builds of targets that cause an import with Auto-Sync. Builds can be scheduled by placing build commands in a batch file and scheduling a call to the batch file with Windows Scheduler.

The target can be a throw-away that builds a one topic TOC. The output doesn’t matter. The target is just a means to get madbuild.exe to cause an import.

But I use source control and madbuild.exe does not prompt me to check files in or out.

Depending on your source control solution, you can automate that also. For example with Team Foundation Server and Visual Studio (Team Explorer), the executable is tf.exe and it is located in a Visual Studio folder.

One thing to watch out for is that not every file in a project should be checked in and out of source control. Some folders are local such as the Output folder in the root of the project and Users folders in the Project folder. The second example that follows shows this.

Build Commands

rem set these for your project
rem set the target down below
set projectfolder="C:\flare-projects\example"
set projectname="example.flprj"
set flarefolder="C:\Program Files (x86)\MadCap Software\MadCap Flare V9\Flare.app"
cd "%flarefolder%"
rem set target here and add other lines as necessary
madbuild.exe -project "%projectfolder%\%projectname%" -target "build-for-import.fltar"
rem madbuild.exe -project "%projectfolder%\%projectname%" -target "build-for-import.fltar"

 

Build with TFS Source Control Commands

rem set these for your project
rem set the target down below
set projectfolder="C:\flare-projects\example"
set projectname="example.flprj"
set visualstudiofolder="C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE"
set flarefolder="C:\Program Files (x86)\MadCap Software\MadCap Flare V9\Flare.app"
cd "%visualstudiofolder%"
tf.exe checkout "%projectfolder%\Content" /recursive /noprompt
tf.exe checkout "%projectfolder%\Project\Advanced" /recursive /noprompt
tf.exe checkout "%projectfolder%\Project\ConditionTagSets" /recursive /noprompt
tf.exe checkout "%projectfolder%\Project\Destinations" /recursive /noprompt
tf.exe checkout "%projectfolder%\Project\Glossaries" /recursive /noprompt
tf.exe checkout "%projectfolder%\Project\Imports" /recursive /noprompt
tf.exe checkout "%projectfolder%\Project\Skins" /recursive /noprompt
tf.exe checkout "%projectfolder%\Project\Targets" /recursive /noprompt
tf.exe checkout "%projectfolder%\Project\TOCs" /recursive /noprompt
tf.exe checkout "%projectfolder%\Project\VariableSets" /recursive /noprompt
cd "%flarefolder%"
rem set target here and add other lines as necessary
madbuild.exe -project "%projectfolder%\%projectname%" -target "build-for-import.fltar"
rem madbuild.exe -project "%projectfolder%\%projectname%" -target "build-for-import.fltar"
cd "%visualstudiofolder%"
tf.exe checkin "%projectfolder%\Content" /recursive /noprompt
tf.exe checkin "%projectfolder%\Project\Advanced" /recursive /noprompt
tf.exe checkin "%projectfolder%\Project\ConditionTagSets" /recursive /noprompt
tf.exe checkin "%projectfolder%\Project\Destinations" /recursive /noprompt
tf.exe checkin "%projectfolder%\Project\Glossaries" /recursive /noprompt
tf.exe checkin "%projectfolder%\Project\Imports" /recursive /noprompt
tf.exe checkin "%projectfolder%\Project\Skins" /recursive /noprompt
tf.exe checkin "%projectfolder%\Project\Targets" /recursive /noprompt
tf.exe checkin "%projectfolder%\Project\TOCs" /recursive /noprompt
tf.exe checkin "%projectfolder%\Project\VariableSets" /recursive /noprompt

 

14 comments

  1. Why does anything have to be checked out of TFS as part of a build? Is it because imports are being used and auto-generated? For a typical build without imports, would it still have to be checked out of source control?

    1. That is it. A build without imports does not have to be checked out. Even one with imports doesn’t have to be checked out. But you’ll notice that nothing is imported in that case. Also a build from a non-source-controlled project that imports from a source controlled project does not require checkouts. Something else to note is that if you have pending adds in a project, these commands will complete those.

  2. This has been very helpful, but we use SVN for or source control. Do you know how to perform the same type of check-out/check-in before and after the build process for SVN? Flare seems to mark files as read-only as part of the check-out/check-in process, but this can’t be the only thing it does, as manually making the files writeable/read-only causes improper file handling once you’re back into the Flare UI (files wont’ show up as pending check-ins, but you can’t edit them either ><)

    Thanks in advance.

    1. I haven’t done it. But it would be an interesting exercise. SVN uses a command line interface, so scripting commits and checkouts in a batch file should be possible. Check out http://svnbook.red-bean.com/en/1.7/svn.ref.svn.c.checkout.html.

  3. I’d like to set up automated imports for my projects, but I’m confused as to why you need the one-topic import target. Doesn’t the fact that the Target has auto-sync enabled and the Import file has auto-reimport enabled accomplish the same thing in the actual project target? Won’t it happen anyway when you build the actual target?

    1. You are correct that a one topic TOC/Target isn’t necessary. That is just an example. If your scenario is that you have a target in a project and you require an import when you build the target, the auto-reimport option on the import and the auto-sync option on the target may have your needs covered. But…

      If your scenario is that you just want to prompt imports into a project using madbuild.exe and you are not concerned with actually using the output of a target with the action, the smallest possible TOC and content is optimal. The reason is because of build times. Suppose you want to import source files such as templates, CSS, and canned text snippets from a master project for templates into twenty other projects. It is possible to rely on auto-sync. But if you only build around the time of publishing, that won’t give authors the latest version of the templates to work with in the mean time. They will either have to run the imports themselves or run a build with auto-sync. If the build takes twenty minutes, you can cut that down with a simpler target. If you are doing that fifty times with twenty minute builds, you can see where that gets to be an issue.

      Here is another scenario. Suppose you have projects with cascading dependencies. I’m not advocating either way about this. But what if you have one project that contains boilerplate text at an organizational level such as copyright notices and company descriptions? And what if you have another set of projects with prepared text for domain-specific items, each project representing a domain? For example one project may have heavily reused user assistance content for one software platform and another may have heavily reused conceptual content for another platform. Now a final set of projects may contain content for various software applications. Each application uses content from a platform project and every application uses content from the organizations project.

      You could import the organizational content (copyright, etc.) into the application (software program) project. Or you could just import it into the platform (used by many software programs) projects and get it into the application projects from an import from those. But the tricky part is the order of imports (organizational > platform > application). That is where scripting the order of imports comes in handy.

      1. Ah, paragraph two makes it click for me, esp with how many projects we need to build. Thank you for the thorough response – I use your blog as a reference quite a bit.

  4. Is there a way to push the import file(s) in the first place using madbuild? I’m thinking all the writers would have to import the import file manually, and then I could push updates to the import file and other “template” files (graphics, skins, targets, etc.) using madbuild. Can I push updates to the import file *and* updated template files at the same time with madbuild , and have the project targets then recognize both the import file and import all the other files in the same madbuild execution?

    1. Something like that would be very useful. Although updating something that defines the update creates a paradox. Design-wise, I wouldn’t agree with MadCap implementing that part. It seems like they haven’t. But you can select Flare Import Files as a type under Include Files in the Project Import Editor. When I tried it with just one import by clicking the Import button in the Project Import Editor or by building a target and relying on auto-synch/auto-reimport, it didn’t work. The import file was not overwritten and it didn’t matter which was edited more recently. However when I added a second Flare Import File to the source project, the import of the second Flare Import File succeeded. Subsequent imports picked up the new file types indicated under Include Files or other changes. But I didn’t test this rigorously. I think it is possible to do most of what you are suggesting but I doubt it could happen in the same execution. Since madbuild.exe builds can be scripted and you don’t have to build a heavy target, is that important? I suggest carefully testing the process before using it for production work. It would also be nice to get some feedback from MadCap about the rules for the this behavior in Flare and madbuild.exe and whether they view those as frozen or something that could change in future versions.

      1. Thanks for taking the time, Thomas. I realize that having an update that defines the updates doesn’t quite make sense, but it seemed like it might be possible the way the feature is designed, and wanted to check my thinking; the scripting and import files are new territory for me. I will reach out to MadCap for their take as well. And I will get back to testing!

        1. Well I like the idea of automatically pulling updates to import files into projects. I think you can pull it off with just the features in the current version. Good luck!

  5. Hi Thomas,

    I’m seeing this post a bit late… I’ve really wanted to make use of the AutoSync in Flare, but for my situation, it doesn’t tend to work well. I have to import content from over 30 different Flare projects into a single, Master project. (Kind of the reverse from most people who import content from one project into multiple ones.)

    We always want to import ALL files from the 30+ projects, even if the import analysis says the file is older. The nature of our working environment means that a lot of times, an outdated file in terms of timestamp is actually the correct one. I’ve found that the AutoSync only imports files that appear to be newer by the timestamp, so it actually misses a lot of important files in the import.

    So, we’ve been doing the imports manually for awhile, but I really want to change that. I was curious if you might have any ideas of working around that restriction. I’ve thought of two “hacks”: (1) altering the timestamps on the files to make them seem newer, but I really don’t want to do that to over 1,000 files or (2) altering the LastImport date of the import file OR the TimeStamp listed for each imported file to something that far outdates our file structure, like 2005. That direction seems most likely to be helpful in terms of the AutoSync, but we would have to update all those .flimpfl files each time. (Not too hard to write up a script/macro for that, but wasn’t sure if that might introduce issues that hadn’t occurred to me.)

    I’d be curious for your thoughts. Thanks!

    1. Sorry about the late response. Thanks for reading! I caught your session at the first MadWorld. I know this isn’t your question. But I’m curious about the order in which you plan to import files. In particular, are there imports or files with the same name and path in those projects? And do you import into one project and then subsequently another, like a cascade of imports?

      In any case, I’ve also run into the timestamp problem. Without having done it recently, at first blush, I prefer the approach you offer for altering the Import file. If you are building an ecosystem of processes around Flare, a .NET application, one technology you might want to consider is Windows PowerShell (http://technet.microsoft.com/en-us/scriptcenter/powershell.aspx). That might make it easier to integrate your solution into a plug-in down the road. But any language with some form of file, string, and XML manipulation would do the trick. Python is popular. But it sounds like you may have one in mind already.

      Now just clearing out those dates may give you a performance hit. My hunch is those are there to save Flare from needlessly re-importing files. But since the feature seems to be excluding based on a way-way-too-aggressive policy, there isn’t much you can do about that unless you make the decision about which files get imported via scripting. That would be an interesting problem to solve. You could go event further and manage the list of files to import with a separate set of files or some kind of database and set those on the Import file prior to every run.

      I’d like to hear more about this.

Leave a comment

Your email address will not be published.

HTML tags are not allowed.

250,806 Spambots Blocked by Simple Comments