Source control and ip cores

I'm looking for a peace of advice.
Currently I use git for version control in my VHDL projects and I usually i=
nclude all .vhd files as well as .xdc constrains files in it. But I don't k=
now what I should do with ip cores. Including the whole directory of ip cor=
e to repository doesn't seem to be a good idea because there are to many fi=
les and too many of them are changed after any change of ip settings. Other=
 choice, manual description of ip settings in comments is tedious and error=
 prone.

On Sun, 07 Feb 2016 00:11:17 -0800, Ilya Kalistru wrote:

> I'm looking for a peace of advice.
> Currently I use git for version control in my VHDL projects and I
> usually include all .vhd files as well as .xdc constrains files in it.
> But I don't know what I should do with ip cores. Including the whole
> directory of ip core to repository doesn't seem to be a good idea
> because there are to many files and too many of them are changed after
> any change of ip settings. Other choice, manual description of ip
> settings in comments is tedious and error prone.

I hear you loud and clear. FPGA tool vendors are a couple of decades 
behind in their understanding of development practices, and often don't 
seem to consider version control at all.

It takes some digging and experimentation to work out which  
configuration files you need to regenerate the IP cores from, and how to 
regenerate them with minimal GUI interaction.

Maybe worth digging around in the "command log" files to see which 
command line tools are run when you press "Generate", and which files 
they take as inputs. That command line can then be extracted for 
scripting the process.

Specific details will depend on the tools of course.

-- Brian

On Sunday, February 7, 2016 at 3:15:25 PM UTC+3, Brian Drummond wrote:
> On Sun, 07 Feb 2016 00:11:17 -0800, Ilya Kalistru wrote:
> 
> > I'm looking for a peace of advice.
> > Currently I use git for version control in my VHDL projects and I
> > usually include all .vhd files as well as .xdc constrains files in it.
> > But I don't know what I should do with ip cores. Including the whole
> > directory of ip core to repository doesn't seem to be a good idea
> > because there are to many files and too many of them are changed after
> > any change of ip settings. Other choice, manual description of ip
> > settings in comments is tedious and error prone.
> 
> I hear you loud and clear. FPGA tool vendors are a couple of decades 
> behind in their understanding of development practices, and often don't 
> seem to consider version control at all.
> 
> It takes some digging and experimentation to work out which  
> configuration files you need to regenerate the IP cores from, and how to 
> regenerate them with minimal GUI interaction.
> 
> Maybe worth digging around in the "command log" files to see which 
> command line tools are run when you press "Generate", and which files 
> they take as inputs. That command line can then be extracted for 
> scripting the process.
> 
> Specific details will depend on the tools of course.
> 
> -- Brian

Ok. That is the approach I'm going to try. It has one disadvantage - configuration files are not really human-readable (xilinx .xci), but it looks like the best way by now.
Thank you for response.

In article <n97cbb$re0$2@dont-email.me>,
Brian Drummond  <brian@shapes.demon.co.uk> wrote:
>On Sun, 07 Feb 2016 00:11:17 -0800, Ilya Kalistru wrote:
>
>> I'm looking for a peace of advice.
>> Currently I use git for version control in my VHDL projects and I
>> usually include all .vhd files as well as .xdc constrains files in it.
>> But I don't know what I should do with ip cores. Including the whole
>> directory of ip core to repository doesn't seem to be a good idea
>> because there are to many files and too many of them are changed after
>> any change of ip settings. Other choice, manual description of ip
>> settings in comments is tedious and error prone.
>
>I hear you loud and clear. FPGA tool vendors are a couple of decades 
>behind in their understanding of development practices, and often don't 
>seem to consider version control at all.

Preach it Brian.  Version control (at least for vendor X) is a late
addition, very poorly bolted on.  The design flows were created with 
an image of a single user owning and maintaining a single FPGA by
themselves.  Very 90's.

>
>It takes some digging and experimentation to work out which  
>configuration files you need to regenerate the IP cores from, and how to 
>regenerate them with minimal GUI interaction.
>

This.  We'll often check in a zip'ed archive of the dummy run we used to create
the IP - along with ALL it's junk.  Just for information purposes.  
As to what's actually USED in FPGA development, we usually peel back all the 
layers of cruft, and find the source RTL under all the chaff.  That's
what's actually used in our scripts / development.   

It's usually much more flexible to use the actual RTL as well, as opposed to 
the crap they bolt on top of it.

Regards,

Mark

Mark Curry <gtwrek@sonic.net> wrote:
> In article <n97cbb$re0$2@dont-email.me>,
> Brian Drummond  <brian@shapes.demon.co.uk> wrote:
> >
> >I hear you loud and clear. FPGA tool vendors are a couple of decades 
> >behind in their understanding of development practices, and often don't 
> >seem to consider version control at all.

I'm also quite tired of vendors trying to re-implement their own version
control systems.  Altium has 'vaults', for instance, which are essentially a
poor interface to Subversion ten years too late.  The files are still binary
and there's no usable merging.  That's no good when your vcs of choice is
git/bzr/hg/...

> This.  We'll often check in a zip'ed archive of the dummy run we used to
> create the IP - along with ALL it's junk.  Just for information purposes. 
> As to what's actually USED in FPGA development, we usually peel back all
> the layers of cruft, and find the source RTL under all the chaff.  That's
> what's actually used in our scripts / development.
> 
> It's usually much more flexible to use the actual RTL as well, as opposed
> to the crap they bolt on top of it.

We have the opposite problem.  As an open source project we cannot ship the
vendor RTL, because it usually says (c) Vendor, Do Not Distribute or
something on it.  But checking in the vendor RTL is the only way to get
things done.

So we have a split workflow: devs check in whatever vendor cruft into the
private repo they need to make things work.  Then, when we want to cut a
public release, we have to go through and work out how to generate the RTL
from licence-clean sources by some invocation of the tools.  Once the
invocations are put in the makefile, we can then build and check things
still work with the generated RTL.  Unfortunately the tools are often not
friendly to this workflow.

Theo

On Sunday, 7 February 2016 21:50:30 UTC+8, Ilya Kalistru  wrote:
> On Sunday, February 7, 2016 at 3:15:25 PM UTC+3, Brian Drummond wrote:
> > On Sun, 07 Feb 2016 00:11:17 -0800, Ilya Kalistru wrote:
> > 
> > > I'm looking for a peace of advice.
> > > Currently I use git for version control in my VHDL projects and I
> > > usually include all .vhd files as well as .xdc constrains files in it.
> > > But I don't know what I should do with ip cores. Including the whole
> > > directory of ip core to repository doesn't seem to be a good idea
> > > because there are to many files and too many of them are changed after
> > > any change of ip settings. Other choice, manual description of ip
> > > settings in comments is tedious and error prone.
> > 
> > I hear you loud and clear. FPGA tool vendors are a couple of decades 
> > behind in their understanding of development practices, and often don't 
> > seem to consider version control at all.
> > 
> > It takes some digging and experimentation to work out which  
> > configuration files you need to regenerate the IP cores from, and how to 
> > regenerate them with minimal GUI interaction.
> > 
> > Maybe worth digging around in the "command log" files to see which 
> > command line tools are run when you press "Generate", and which files 
> > they take as inputs. That command line can then be extracted for 
> > scripting the process.
> > 
> > Specific details will depend on the tools of course.
> > 
> > -- Brian
> 
> Ok. That is the approach I'm going to try. It has one disadvantage - configuration files are not really human-readable (xilinx .xci), but it looks like the best way by now.
> Thank you for response.

Xilinx's configuration files aren't really that human-unreadable. I am actually amazed at how much standards support Xilinx puts into their tool and flows. If you take a good look into an *.xci file with a text editor, you will notice that it is in a human-readable XML format. And this is no ordinary XML - the format of the XML actually adheres to the IP-XACT specification, which is indeed a portable method to package and transfer components between tools that support this specification.
https://en.wikipedia.org/wiki/IP-XACT

-daniel

On Tuesday, 23 February 2016 20:38:17 UTC+8, Daniel Kho  wrote:
> On Sunday, 7 February 2016 21:50:30 UTC+8, Ilya Kalistru  wrote:
> > On Sunday, February 7, 2016 at 3:15:25 PM UTC+3, Brian Drummond wrote:
> > > On Sun, 07 Feb 2016 00:11:17 -0800, Ilya Kalistru wrote:
> > > 
> > > > I'm looking for a peace of advice.
> > > > Currently I use git for version control in my VHDL projects and I
> > > > usually include all .vhd files as well as .xdc constrains files in it.
> > > > But I don't know what I should do with ip cores. Including the whole
> > > > directory of ip core to repository doesn't seem to be a good idea
> > > > because there are to many files and too many of them are changed after
> > > > any change of ip settings. Other choice, manual description of ip
> > > > settings in comments is tedious and error prone.
> > > 
> > > I hear you loud and clear. FPGA tool vendors are a couple of decades 
> > > behind in their understanding of development practices, and often don't 
> > > seem to consider version control at all.
> > > 
> > > It takes some digging and experimentation to work out which  
> > > configuration files you need to regenerate the IP cores from, and how to 
> > > regenerate them with minimal GUI interaction.
> > > 
> > > Maybe worth digging around in the "command log" files to see which 
> > > command line tools are run when you press "Generate", and which files 
> > > they take as inputs. That command line can then be extracted for 
> > > scripting the process.
> > > 
> > > Specific details will depend on the tools of course.
> > > 
> > > -- Brian
> > 
> > Ok. That is the approach I'm going to try. It has one disadvantage - configuration files are not really human-readable (xilinx .xci), but it looks like the best way by now.
> > Thank you for response.
> 
> Xilinx's configuration files aren't really that human-unreadable. I am actually amazed at how much standards support Xilinx puts into their tool and flows. If you take a good look into an *.xci file with a text editor, you will notice that it is in a human-readable XML format. And this is no ordinary XML - the format of the XML actually adheres to the IP-XACT specification, which is indeed a portable method to package and transfer components between tools that support this specification.
> https://en.wikipedia.org/wiki/IP-XACT
> 
> -daniel

Yes, I think it is a good idea to version control IP-XACT files.

-daniel

Ilya Kalistru <stebanoid@gmail.com> wrote:
> I'm looking for a peace of advice.
> Currently I use git for version control in my VHDL projects and I usually
> include all .vhd files as well as .xdc constrains files in it. But I don't
> know what I should do with ip cores. Including the whole directory of ip core
> to repository doesn't seem to be a good idea because there are to many files
> and too many of them are changed after any change of ip settings. Other
> choice, manual description of ip settings in comments is tedious and error
> prone.

I check in only the xci and xml files in the sources_1/ip structure.

When checking out a pristine tree from source control and run generate
bitstream for the first time, Vivado will go through the process of
regenerating all the products of the included IP.

This is GUI workflow.

-- 
svenn

In article <4b7d68c4-7824-40d8-a114-b0c331b979a2@googlegroups.com>,
Daniel Kho  <daniel.kho@gmail.com> wrote:
>On Sunday, 7 February 2016 21:50:30 UTC+8, Ilya Kalistru  wrote:
>> On Sunday, February 7, 2016 at 3:15:25 PM UTC+3, Brian Drummond wrote:
>> > On Sun, 07 Feb 2016 00:11:17 -0800, Ilya Kalistru wrote:
>> > 
>> > > I'm looking for a peace of advice.
>> > > Currently I use git for version control in my VHDL projects and I
>> > > usually include all .vhd files as well as .xdc constrains files in it.
>> > > But I don't know what I should do with ip cores. Including the whole
>> > > directory of ip core to repository doesn't seem to be a good idea
>> > > because there are to many files and too many of them are changed after
>> > > any change of ip settings. Other choice, manual description of ip
>> > > settings in comments is tedious and error prone.
>> > 
>> > I hear you loud and clear. FPGA tool vendors are a couple of decades 
>> > behind in their understanding of development practices, and often don't 
>> > seem to consider version control at all.
>> > 
>> > It takes some digging and experimentation to work out which  
>> > configuration files you need to regenerate the IP cores from, and how to 
>> > regenerate them with minimal GUI interaction.
>> > 
>> > Maybe worth digging around in the "command log" files to see which 
>> > command line tools are run when you press "Generate", and which files 
>> > they take as inputs. That command line can then be extracted for 
>> > scripting the process.
>> > 
>> > Specific details will depend on the tools of course.
>> > 
>> > -- Brian
>> 
>> Ok. That is the approach I'm going to try. It has one disadvantage - configuration 
>> files are not really human-readable (xilinx .xci), but it looks like the best way 
>> by now.
>
>Xilinx's configuration files aren't really that human-unreadable. I am actually amazed at 
>how much standards support Xilinx puts into their tool and flows. If you take a good look
>into an *.xci file with a text editor, you will notice that it is in a human-readable XML
>format. And this is no ordinary XML - the format of the XML actually adheres to the 
>IP-XACT specification, which is indeed a portable method to package and transfer components 
>between tools that support this specification.
>https://en.wikipedia.org/wiki/IP-XACT

(Getting ranty, and not helping the OP, but this is a sore spot for us...)

Xilinx support of standards is 2 steps forwards, 1.9 steps back.  Sure they're using 
more "industry standard" files.  (SDC, TCL).  But they're still burying the landscape
in a host of unneccesary, undocumented, randomly changing list of other ancillary 
files. 

I'm really not clear as to what IP-XACT is trying to solve, and who it's audience is.
But XCI files are a very unreliable way of managing IP.  Sure it's XML - there's
tools available to manipulate them.  But Xilinx doesn't document the
data within those files in any way.  They're free to change it at any time.
Can a user reliably MERGE changes within those files from one development
tree, and another?  Will the design still build 1, 2, 5, or 10 years 
down the line using that undocumented src file?

Can you simulate an XCI file?  Can you run lint on it?  Can you modify it's contents?
What's configurable within them?  In the end, it's just something that gets in the way
of what you really need to do those tasks - the RTL (or netlist as neccesary - 
encrypted at worst case).

Regards,

Mark

I'm working at a big project with multiple developers and we also want to be able to share sources in a version controlled friendly.
The project mainly uses X... FPGAs and V... tools.

One of possible approaches was to use IP packager, however it seems that it is not possible, or at least discouraged to use IP cores with port defined as VHDL records (just imagine a few AXI ports as bits and bit vectors, and compare it to the clean design of IPbus with record based ports - https://svnweb.cern.ch/trac/cactus )

The another approach I've proposed is based on extension of the old good PRJ files.

The file may contain lines of following types:
  - type library file
    (Line defining the source file
    Possible types: vhdl, verilog, system, header. xci, xdc
    File name (path in fact) is defined relatively to the location of the
    PRJ file)

  - include file
    (Line defining the include file
File name (path in fact) is defined relatively to the location of the
PRJ file
Use of relative paths allows easy reuse of so defined
blocks. Use of simple text files makes such solution
SVN/GIT friendly