A limitation of XST requires a stacksize proportionate to the depth of the design.
Starting from ISE Design Suite 14.1, XST will generate the following error message if you have a stacksize that is limited:
This message has been added in 14.1 XST to direct the users to this answer record.
The above message means that the stacksize that is currently set is not sufficient for XST to complete the synthesis process.
The stacksize should be changed to unlimited by using the following commands:
if the shell is sh, bash or ksh use:
ulimit -s unlimited
if the shell is csh, tcsh or zsh use:
limit stacksize unlimited
Run the following command on the Linux/Unix shell for checking the stacksize:
The output of the above command should resemble the following message:
Now, run XST again and this should fix the FATAL ERROR message.
If for some reason there is a restriction on the stacksize or XST is running on Windows, then add the following lines to the xst file and run XST again:
Setting the above variables alters the behavior of XST and can degrade QoR.
The recommended flow is to increase the stacksize.