Why does Xilinx hate version control?

jeverett@xilinx.com wrote:
>>>Regarding creating a batch file from the GUI, you can just cut and paste
>>>the
>>>commands from the command_log file.
>>
>>Why should the (skilled) user have to trawl these log files ?
>>
>>It would be nice to  configure via GUI, and then be able to
>>run a created batch file, and get an indentical build.
> 
> 
> 
> No trawling for the command_log file needed.  All you need to do is:
> 
> 1) Run the process Design Utilities ->"View Command Line Log File"
>     this copies the latest command lines to the ISE Text Editor
> 
> 2) Just File->Save As...  mydesign.bat and you have your batch file
> 
> 3) Open a command prompt and run the bat file

Sounds tolerable - does that give an identical build ?

Is there any way to init the GUI, from such a log, should
you wish to change one setting in 6 months time ?

-jg

Jake Janovetz wrote:
> 
> I, too, prefer command line tools for many things.  Unfortunately, when
> I moved to Windows out of necessity (CAD software mainly), I realized
> that the environment is just CL-hostile.

That's why god invented Linux.

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo
+-----------------------------------------------------------+
"Perl as a language has less a design than a thousand special features
flying in close formation."
-- From the c2 wiki

Jim Granville wrote:

> Hi Steve,
>   Good to hear from someone in Xilinx
> 
> Steve Lass wrote:
> 
>> First, let me try to address the reason why the ISE file is binary.  A 
>> binary file
>> allows us to manage concurrent reads/writes which is critical in 
>> making all
>> the GUI applications work together.  Binary files are faster and more 
>> efficient.
>> Right now, there is a bug that is making the ISE file much larger than 
>> it needs to be.  
> 
> 
> So how can _that_ be faster and more efficent ? Did no one notice this ?
> Such an error would have been spotted very quickly with ASCII files....
I will not claim to know all the details, but suffice it to say, if the 
ncd was
ASCII, runtimes would be much longer.
> 
> Also, a binary file can be more robust and requires less error
> 
>> checking.
> 
> 
> Yeeessss, only until said file gets corrupted, and then it becomes a 
> brick wall. Moving across tool versions also a risk area, and likely
> to be a minefield....
> 
>>
>> Access to the data in the ISE file is often important, so providing 
>> the capability
>> to import and export is key.  Check answer record 21067 for info on 
>> how to
>> do this.  Other than the GUI, the standard way to add info into the 
>> ISE file is
>> Tcl, however, 8.2i will be required for this capability.
> 
> 
> and recovery from a corrupted file is done the same way ?
> 
>  Seems to me if you must use binary for your convenience, that you 
> should also provide an easy ASCII import/export as well.
I agree 100%.
> 
>  That way, users CAN archive  a 100% ASCII project ( and some (most?) 
> WILL prefer to do this ), but they can also reap the benefits(?) of 
> binary files during re-iterations.
> 
>>

> When is 8,2i due for release ?
June.

Steve
> 
> -jg
> 

Steve Lass wrote:
> First, let me try to address the reason why the ISE file is binary.  A
> binary file
> allows us to manage concurrent reads/writes which is critical in making all
> the GUI applications work together.  Binary files are faster and more
> efficient.

"Faster and more efficient?"  We're using fairly fast machines and
while I understand that parsing a text file might be inefficient, it's
certainly not going to be the limiting factor when compiling, placing
and routing.

Being able to hand-edit and diff a configuration file, as well as
adding comments to one, far outweighs any slight performance hits.

> Right now, there is a bug that is making the ISE file much larger than
> it needs
> to be.  Also, a binary file can be more robust and requires less error
> checking.

I would imagine that it's easier for the human being who needs to use
the project file to spot and fix errors if the file is plain text.

What sort of errors are we talking about?  It's a binary file now, and
as such can only be created and modified by your tools.  If there's a
problem in the file, it was probably caused by a tool bug.

> Regarding keeping intermediate files in a separate directory, that is a
> great
> idea.  We are planning on allowing you to specify the directory structure in
> the future.

If Xilinx has just now realized this is a great idea, then it's clear
that you don't eat your own dog food.  (Sorry for reusing this
metaphor, but it's apropos.)

> Regarding creating a batch file from the GUI, you can just cut and paste the
> commands from the command_log file.
>
> We do not hate version control and have plans to allow for integration with
> your source control systems.

Subversion!  Subversion!  Subversion!

-a

Steve Lass wrote:

> I will not claim to know all the details, but suffice it to say, if the
> ncd was ASCII, runtimes would be much longer.

The NCD is an intermediate file created by the tools and I wouldn't put
it into my repository.

The files that need to be in revision control are the HDL sources
(obviously), constraint files (UCF remains, thankfully, plain text),
and a Makefile or the project file.  In other words, anything that's
used to drive the build process.   These files need to be plain text so
they can be diffed.  But by all means, intermediate files should be
considered transient, and should be designed for efficiency.  (And put
into their own directory so they can be deleted easily.)

Note that binaries and build results are not normally kept in the
repository.  However, when I tag a design build for release, I include
the .mcs or .jed or whatever is used to actually program the part.
Tags are (either by agreement or enforced) write-once then read-only
and immutable.

-a

Steve Lass <lass@xilinx.com> wrote:
>First, let me try to address the reason why the ISE file is binary.  A 
>binary file allows us to manage concurrent reads/writes which is critical
>in making all the GUI applications work together.  Binary files are faster
>and more efficient.

If IPC (InterProcess Communication) is needed, why not use shared memory or
pipes (local stream) ..?

And such files if they need to exist should be seperate from any kind of
source data.

>Right now, there is a bug that is making the ISE file much larger than 
>it needs to be.  Also, a binary file can be more robust and requires less
>error checking.

Binary files are hard to rescue for anyone without access to the sourcecode.

If specification on how to generate the bitstream(s) were available.
It would enable independent developers to write tools. And avoid the issues
discussed to some extent.

>Access to the data in the ISE file is often important, so providing the 
>capability to import and export is key.  Check answer record 21067 for
>info on how to do this.  Other than the GUI, the standard way to add
>info into the ISE file is
>Tcl, however, 8.2i will be required for this capability.

>Regarding keeping intermediate files in a separate directory, that is a 
>great idea.  We are planning on allowing you to specify the directory
>structure in the future.

This would be very useful for complex setups.

>Regarding creating a batch file from the GUI, you can just cut and paste the
>commands from the command_log file.

A configuration file would be better. To allow version control, loading from
a previous setup - modify and save.

>We do not hate version control and have plans to allow for integration with
>your source control systems.

What will this integration mean in practice ?
(so that potential users may give input)

>We are listening and taking your input seriously.

Not all corporations listen to their customers, so it's nice to see a positive
approach.

   Regards
    /Peter

On Wed, 15 Mar 2006 11:14:28 -0800, Andy Peters wrote:

> Steve Lass wrote:
> 
>> I will not claim to know all the details, but suffice it to say, if the
>> ncd was ASCII, runtimes would be much longer.
> 
> The NCD is an intermediate file created by the tools and I wouldn't put
> it into my repository.
> 
> The files that need to be in revision control are the HDL sources
> (obviously), constraint files (UCF remains, thankfully, plain text), and
> a Makefile or the project file.  In other words, anything that's used to
> drive the build process.   These files need to be plain text so they can
> be diffed.  But by all means, intermediate files should be considered
> transient, and should be designed for efficiency.  (And put into their
> own directory so they can be deleted easily.)
> 
> Note that binaries and build results are not normally kept in the
> repository.  However, when I tag a design build for release, I include
> the .mcs or .jed or whatever is used to actually program the part. Tags
> are (either by agreement or enforced) write-once then read-only and
> immutable.
> 
> -a

The 8.1 tools can still import the old .npl format, the problem is that
they can't save a new project in the .npl format. If Xilinx would add an
export to npl command I think everyone will be happy.

There was no good reason to switch from an ascii file to a binary file
because an ascii file is so much more flexible. The argument that there is
a performance advantage to a binary project file is ridiculous,
translating an ascii file to an internal format takes milliseconds. The
advantages of an ascii file are huge. First off you can read it and see
how everything is set. Secondly you can edit it, it's much easier to do
things in Emacs then it is in any GUI. And finally you can generate them.

I never generate ISE projects by hand. I do all of my designs with
HDLmaker, http://www.polybus.com/hdlmaker/users_guide/, which generates
scripts and make files for simulation (NCVerilog, ModelSim, VCS), scripts
and project files for synthesis (XST, Synplify, and Precision), and
project files for ISE (.npl) and for Quartus (.qpf and .qsf).

Although I almost never use the GUI, the one exception is in the lab when
running ChipScope, I have customers who are more comfortable with a GUI
particularly those who are still using Windows. For them I need to be able
to generate a project file. Fortunately .npl format still works but if
Xilinx ever drops support for the .npl format it won't be possible to
generate a project file anymore.

Andy Peters wrote:
> These files need to be plain text so
> they can be diffed.  

Not necessarily. Xilinx could also provide a diff tool for the file
format. Of course with a FOSS source license so that you can create a
plugin for the versioning system of your choice.
But I agree that ASCII is the simpler way to go.

Steve Lass wrote:
> A binary file allows us to manage concurrent reads/writes which is
> critical in making all the GUI applications work together.

Maybe you should split into multiple files. Your comments sounds as if
the tools interact a lot via the project files. That is not something
that belongs into a repository.
What is needed there is mainly information about what files belong to
the project and the project settings selected by the user. This is only
a few hundred bytes. A lot of that information is ascii strings anyway.

Actually I do not see why concurrent read writes are easier in binary
files than in ascii files. Fixed field width vs. dynamic field width
makes a difference. But you can have fixed width ascii files as well.

Kolja Sulimma

Hi Steve-

Thanks for reading and responding!

Re: binary files...  Binary files have their advantages.  However, the 
rest of the software industry has recognized their distinct 
disadvantages in many cases and have moved in droves to XML.  It's 
really quite beautiful in its simplicity.  May I suggest that the ISE 
tools store XML for SCM purposes, but (if they want to), create a binary 
during operation for cross-app integration.

The ISE file should really only contain those parameters necessary to 
describe a project and its components.  It should not contain temp 
results and should not even contain results of a project build (like 
statistics).  Place those in another file (pref. text/HTML) and let the 
user decide if she wants to add them to SCM or not.  They definitely 
don't need to be added if a minor update has been made.

Please consider the separate build directory.  I've suggested it through 
different channels over the past few years.  Deaf ears, I suppose. 
Visual Studio does this (as do many other tools) and also stores project 
info in XML.  Temp files are binary (like precompiled headers and 
Intellisense info)

    Jake


Steve Lass wrote:
> First, let me try to address the reason why the ISE file is binary.  A 
> binary file
> allows us to manage concurrent reads/writes which is critical in making all
> the GUI applications work together.  Binary files are faster and more 
> efficient.
> Right now, there is a bug that is making the ISE file much larger than 
> it needs
> to be.  Also, a binary file can be more robust and requires less error 
> checking.
> 
> Access to the data in the ISE file is often important, so providing the 
> capability
> to import and export is key.  Check answer record 21067 for info on how to
> do this.  Other than the GUI, the standard way to add info into the ISE 
> file is
> Tcl, however, 8.2i will be required for this capability.
> 
> Regarding keeping intermediate files in a separate directory, that is a 
> great
> idea.  We are planning on allowing you to specify the directory 
> structure in
> the future.
> 
> Regarding creating a batch file from the GUI, you can just cut and paste 
> the
> commands from the command_log file.
> 
> We do not hate version control and have plans to allow for integration with
> your source control systems.
> 
> We are listening and taking your input seriously.
> 
> Regards,
> 
> Steve
> 
> Jake Janovetz wrote:
>> Is there some internal Xilinx conspiracy against source code 
>> management like SVN (subversion) and CVS?  Or is it that the Xilinx 
>> guys don't use version control to understand the goals?
>>
>> ISE 6.x used ".npl" files to contain the project information.  These 
>> were text-based making them at least somewhat SCM-friendly, but they 
>> changed each and every time you saved the project even if nothing 
>> changed.  Some date code changed.  Thus requiring an update...
>>
>> ISE 7.x came along and, even when the rest of the world was switching 
>> to XML because of all the problems with binary config files, Xilinx 
>> decided to move to a binary format ".ise" from it's .npl files.  Now, 
>> each SCM checkin required the whole binary file to be checked in each 
>> time rather than just diffs (like the ISE 6.x days).
>>
>> ISE 8.x came along and the conspiracy became clearer.  Xilinx held on 
>> to its binary format but has apparently added a LOT more to the file.  
>> Now, it's almost 1 MB!!!  This means that my SCM repository grows by 1 
>> MB EACH TIME I do a checkin if I include the ISE file.  That's 
>> ridiculous!
>>
>>
>> PLEASE Xilinx, be learn about CVS, SVN, and others, and how to design 
>> file formats for SCM.  Also, place all temporary files in a temp 
>> directory and stop spamming my project directory.  Oh, and one more 
>> thing -- it would be nice to know which files from a CORE are 
>> necessary to the project.  Each CORE generates almost a dozen files 
>> and I'd rather not add all of them to SCM.
>>
>>    Jake
>