Showing posts with label ssis. Show all posts
Showing posts with label ssis. Show all posts

Monday, March 26, 2012

how to share variables across packages?

I'm working on a solution in Visual Studio that has 3 SSIS packages, and now I want to add a 4th that needs access to variables defined in one of the other packages. How can I do that?

One way would be to dump out your variables and values to a raw file or db if you prefer. Then any other process from any execution context can access them anytime via a "source connection" to that file...

|||

Kevin,

Have you looked at package configurations at all. You could set parameter values from a table or a parent package variable.

Rafael Salas

|||

Rafael Salas wrote:

Have you looked at package configurations at all. You could set parameter values from a table or a parent package variable.

Yes, I intend to set my variables externally, via package configurations. But I don't know what a "parameter" is or how it differs from a variable. And I don't know how to allow my variables scoped to package X to be accessed by package Y -- how do I make package X to be a parent of package Y?

|||

Sorry I meant variable...not parameter.

In short, to make X parent of Y; you can use a Execute Package Task (control Flow) in X that would call package Y.

Rafael Salas

Friday, March 9, 2012

How to Set Record Length for the Flatfiles in SSIS

I am trying to transfer the data from OLEDB source to Flatfile destination. My Client need the flatfile of Record Length 80 Bytes. What does this means? Does it mean the output width or does it mean the flatfile format should be fixed length? please explain?It means that the sum of the individual column lengths must add up to 80. No less, no more.

You need to find out, though, if there is supposed to be a line feed at the end of each row, in which case you'd want to use a "ragged-right" format. Otherwise, fixed width would work.|||so if i set the output column width of each column to 80, it is all set, as the client dont have further data available.....he does not knw about the line feed......|||No, the columns must ALL add up, in total, to 80.|||great thanks phil.....I think this answers my question for now....i will keep the thread open as i might come up with more questions on this topic|||

one thing more is, if have set the record length for all the columns (total = 80 bytes), now when i name the columns should that also comply with number of bits i have set for each column

for eg

if columnname is MyNumber itz datataype is Varchar and Length is 10 than do MyNumber also be set within 10 Bits limit

|||

B.Chintan wrote:

one thing more is, if have set the record length for all the columns (total = 80 bytes), now when i name the columns should that also comply with number of bits i have set for each column

for eg

if columnname is MyNumber itz datataype is Varchar and Length is 10 than do MyNumber also be set within 10 Bits limit

It depends. If you are outputting column names to the file, then yes. (Even that might not be true, but in general, it is.) If you are not outputting file names to the file, then you can make them however long you need.

|||

I tried the above stated solution, when the column name is not specified it works fine and when column name is specified, it does not work as it wants the names to be within 80 bytes limit.

The problem now is when i try to put this flat files back to database using oledbsource and lookups, I am not able to do that. i encounter problem with format.

In this case i have set the format to fixed width...so when i try to put it in db.....i can see just one column whereas when i converted it to flatfile it had 61 columns and now while reversing it i c everything in one column.

Any step by step procedure to solve this tht is from db to flatfile with fixed width row dilimiter and vice versa

Chintan Shah

|||Then make your column names fit into the 80 bytes length restriction.

Also, use "ragged right" when loading the resulting flat file. But why are you doing this anyway? Just to test the file?|||i need to submit the flat files to an reporting agency now they have certain standards which my client need to follow, putting the flat file back to db is just a test I am doing as tht will be done by the agency when they receive the flat files from us. I will try using ragged right....hope tht works fine.|||

the best solution i did was to transform the db file in 3 format using this

OLEDBSource>>>>MultiCast>>>>3 Diff flatfiles in 3 diff formats.

One important thing to remember is

If the column headings are to be submitted than they should comply to the set Byte Limit. If they are not than column headings can be as long as desired.

Thanks phil for all help.

I would like to know if you have any Knowledge about Microsoft Dynamics and is it a good line to enter or SQL Server is the better option.

Chintan

How to Set Record Length for the Flatfiles in SSIS

I am trying to transfer the data from OLEDB source to Flatfile destination. My Client need the flatfile of Record Length 80 Bytes. What does this means? Does it mean the output width or does it mean the flatfile format should be fixed length? please explain?It means that the sum of the individual column lengths must add up to 80. No less, no more.

You need to find out, though, if there is supposed to be a line feed at the end of each row, in which case you'd want to use a "ragged-right" format. Otherwise, fixed width would work.|||so if i set the output column width of each column to 80, it is all set, as the client dont have further data available.....he does not knw about the line feed......|||No, the columns must ALL add up, in total, to 80.|||great thanks phil.....I think this answers my question for now....i will keep the thread open as i might come up with more questions on this topic|||

one thing more is, if have set the record length for all the columns (total = 80 bytes), now when i name the columns should that also comply with number of bits i have set for each column

for eg

if columnname is MyNumber itz datataype is Varchar and Length is 10 than do MyNumber also be set within 10 Bits limit

|||

B.Chintan wrote:

one thing more is, if have set the record length for all the columns (total = 80 bytes), now when i name the columns should that also comply with number of bits i have set for each column

for eg

if columnname is MyNumber itz datataype is Varchar and Length is 10 than do MyNumber also be set within 10 Bits limit

It depends. If you are outputting column names to the file, then yes. (Even that might not be true, but in general, it is.) If you are not outputting file names to the file, then you can make them however long you need.

|||

I tried the above stated solution, when the column name is not specified it works fine and when column name is specified, it does not work as it wants the names to be within 80 bytes limit.

The problem now is when i try to put this flat files back to database using oledbsource and lookups, I am not able to do that. i encounter problem with format.

In this case i have set the format to fixed width...so when i try to put it in db.....i can see just one column whereas when i converted it to flatfile it had 61 columns and now while reversing it i c everything in one column.

Any step by step procedure to solve this tht is from db to flatfile with fixed width row dilimiter and vice versa

Chintan Shah

|||Then make your column names fit into the 80 bytes length restriction.

Also, use "ragged right" when loading the resulting flat file. But why are you doing this anyway? Just to test the file?|||i need to submit the flat files to an reporting agency now they have certain standards which my client need to follow, putting the flat file back to db is just a test I am doing as tht will be done by the agency when they receive the flat files from us. I will try using ragged right....hope tht works fine.|||

the best solution i did was to transform the db file in 3 format using this

OLEDBSource>>>>MultiCast>>>>3 Diff flatfiles in 3 diff formats.

One important thing to remember is

If the column headings are to be submitted than they should comply to the set Byte Limit. If they are not than column headings can be as long as desired.

Thanks phil for all help.

I would like to know if you have any Knowledge about Microsoft Dynamics and is it a good line to enter or SQL Server is the better option.

Chintan

How to Set Record Lenght for the Flatfiles in SSIS

I am trying to transfer the data from OLEDB source to Flatfile destination. My Client need the flatfile of Record Length 80 Bytes. What does this means? Does it mean the output width or does it mean the flatfile format should be fixed length? please explain?It means that the sum of the individual column lengths must add up to 80. No less, no more.

You need to find out, though, if there is supposed to be a line feed at the end of each row, in which case you'd want to use a "ragged-right" format. Otherwise, fixed width would work.|||so if i set the output column width of each column to 80, it is all set, as the client dont have further data available.....he does not knw about the line feed......|||No, the columns must ALL add up, in total, to 80.|||great thanks phil.....I think this answers my question for now....i will keep the thread open as i might come up with more questions on this topic|||

one thing more is, if have set the record length for all the columns (total = 80 bytes), now when i name the columns should that also comply with number of bits i have set for each column

for eg

if columnname is MyNumber itz datataype is Varchar and Length is 10 than do MyNumber also be set within 10 Bits limit

|||

B.Chintan wrote:

one thing more is, if have set the record length for all the columns (total = 80 bytes), now when i name the columns should that also comply with number of bits i have set for each column

for eg

if columnname is MyNumber itz datataype is Varchar and Length is 10 than do MyNumber also be set within 10 Bits limit

It depends. If you are outputting column names to the file, then yes. (Even that might not be true, but in general, it is.) If you are not outputting file names to the file, then you can make them however long you need.

|||

I tried the above stated solution, when the column name is not specified it works fine and when column name is specified, it does not work as it wants the names to be within 80 bytes limit.

The problem now is when i try to put this flat files back to database using oledbsource and lookups, I am not able to do that. i encounter problem with format.

In this case i have set the format to fixed width...so when i try to put it in db.....i can see just one column whereas when i converted it to flatfile it had 61 columns and now while reversing it i c everything in one column.

Any step by step procedure to solve this tht is from db to flatfile with fixed width row dilimiter and vice versa

Chintan Shah

|||Then make your column names fit into the 80 bytes length restriction.

Also, use "ragged right" when loading the resulting flat file. But why are you doing this anyway? Just to test the file?|||i need to submit the flat files to an reporting agency now they have certain standards which my client need to follow, putting the flat file back to db is just a test I am doing as tht will be done by the agency when they receive the flat files from us. I will try using ragged right....hope tht works fine.|||

the best solution i did was to transform the db file in 3 format using this

OLEDBSource>>>>MultiCast>>>>3 Diff flatfiles in 3 diff formats.

One important thing to remember is

If the column headings are to be submitted than they should comply to the set Byte Limit. If they are not than column headings can be as long as desired.

Thanks phil for all help.

I would like to know if you have any Knowledge about Microsoft Dynamics and is it a good line to enter or SQL Server is the better option.

Chintan

How to set ProhibitDtd in SSIS?

Hi,

I'm having a problem using XML-files as source for dataflow-task.

On the source-editor for XML i chose the XML-file and the XSD-file. After clicking OK SSIS throws an error, saying I need to set ProhibitDtd in XmlReaderSettings to false, otherwise I'm not able to work with this file.

Unfortunatley I dont know were I can set this option.

Regards,

Jan Wagner

It sounds like your source XML file contains embedded DTD. If this is true, try to remove the DTD first.

Regards,
Yitzhak

|||

I'm having the same problem using XML-files as source for dataflow-task.

My XML has an external DTD so removing it from the source is not an option.

So the question is "How can I set the ProhibitDTD in the XMLReaderSettings to false for SSIS processing?"..

|||

Thanks for your answer Yitzhak.

It just seems I mistook DTD with XSD...

Regards,

Jan

|||

Unfortunately, SSIS doesn't expose all existing XML settings.

I don't know if it is possible for you to do.
Try to change the input XML document as a URI string (http://...) instead of the IXPathNavigable (drive:\...)
My guess is when you do that the following is what's happening inside:

XslCompiledTransform creates an XmlReader to read the XML document from this URI using default XmlReaderSettings, but allowing Document Type Definition (DTD):

XmlReaderSettings rs = new XmlReaderSettings();

rs.ProhibitDtd = false;

XmlReader.Create(inputUri, rs);

Regards,
Yitzhak

How to set ProhibitDtd in SSIS?

Hi,

I'm having a problem using XML-files as source for dataflow-task.

On the source-editor for XML i chose the XML-file and the XSD-file. After clicking OK SSIS throws an error, saying I need to set ProhibitDtd in XmlReaderSettings to false, otherwise I'm not able to work with this file.

Unfortunatley I dont know were I can set this option.

Regards,

Jan Wagner

It sounds like your source XML file contains embedded DTD. If this is true, try to remove the DTD first.

Regards,
Yitzhak

|||

I'm having the same problem using XML-files as source for dataflow-task.

My XML has an external DTD so removing it from the source is not an option.

So the question is "How can I set the ProhibitDTD in the XMLReaderSettings to false for SSIS processing?"..

|||

Thanks for your answer Yitzhak.

It just seems I mistook DTD with XSD...

Regards,

Jan

|||

Unfortunately, SSIS doesn't expose all existing XML settings.

I don't know if it is possible for you to do.
Try to change the input XML document as a URI string (/) instead of the IXPathNavigable (drive:\...)
My guess is when you do that the following is what's happening inside:

XslCompiledTransform creates an XmlReader to read the XML document from this URI using default XmlReaderSettings, but allowing Document Type Definition (DTD):

XmlReaderSettings rs = new XmlReaderSettings();

rs.ProhibitDtd = false;

XmlReader.Create(inputUri, rs);

Regards,
Yitzhak

How to set ProhibitDtd in SSIS?

Hi,

I'm having a problem using XML-files as source for dataflow-task.

On the source-editor for XML i chose the XML-file and the XSD-file. After clicking OK SSIS throws an error, saying I need to set ProhibitDtd in XmlReaderSettings to false, otherwise I'm not able to work with this file.

Unfortunatley I dont know were I can set this option.

Regards,

Jan Wagner

It sounds like your source XML file contains embedded DTD. If this is true, try to remove the DTD first.

Regards,
Yitzhak

|||

I'm having the same problem using XML-files as source for dataflow-task.

My XML has an external DTD so removing it from the source is not an option.

So the question is "How can I set the ProhibitDTD in the XMLReaderSettings to false for SSIS processing?"..

|||

Thanks for your answer Yitzhak.

It just seems I mistook DTD with XSD...

Regards,

Jan

|||

Unfortunately, SSIS doesn't expose all existing XML settings.

I don't know if it is possible for you to do.
Try to change the input XML document as a URI string (http://...) instead of the IXPathNavigable (drive:\...)
My guess is when you do that the following is what's happening inside:

XslCompiledTransform creates an XmlReader to read the XML document from this URI using default XmlReaderSettings, but allowing Document Type Definition (DTD):

XmlReaderSettings rs = new XmlReaderSettings();

rs.ProhibitDtd = false;

XmlReader.Create(inputUri, rs);

Regards,
Yitzhak

Wednesday, March 7, 2012

How to set limit in SSIS

Hi,

I am trying to import some 35-40 flat files into my SQL Database, however I don't want to imort all the data from each file, but just top 20 rows only for data quality check. Flat files are very huge so I don't want to import all of the data into my database.

Thanks,

gpat

gvphubli wrote:

Hi,

I am trying to import some 35-40 flat files into my SQL Database, however I don't want to imort all the data from each file, but just top 20 rows only for data quality check. Flat files are very huge so I don't want to import all of the data into my database.

Thanks,

gpat

Does this help?

Select Top N in a data-flow

(http://blogs.conchango.com/jamiethomson/archive/2005/07/27/SSIS-Nugget_3A00_-Select-Top-N-in-a-data_2D00_flow.aspx)

-Jamie

Friday, February 24, 2012

how to set connection string value by using SET switch of dtexec sql server 2005 command

hi

I need to load a text file into sql server table using SSIS package.

the idea is to load this file with the help of dtexec command by giving the file path and name in the SET switch of dtexec command.

anyone having an idea; would be of great help.

Regards,

Salman Shehbaz.

Try using the package configuration wizard to get the property path:

Jamie talks about it here:

http://blogs.conchango.com/jamiethomson/archive/2007/03/13/SSIS_3A00_-Property-Paths-syntax.aspx

|||

o.kays

i finally got hold of the command;

here it is

execute master.dbo.xp_cmdshell 'dtexec /Ser "ServerName" /SQ "SSISPackagehName" /SET "\Package.Connections[ConnectionManagerName].Properties[ConnectionString]";"TextFilePath/Name"'

Regards,

Salman Shehbaz.

Sunday, February 19, 2012

how to separate date & time during export

I would like to export sql server 2005 tables to a text file using SSIS. It is easy to do this as a direct export. However, how could I separate the datetime field into two fields: one for date only and one for time only?

Sample:

select getdate() ==> 2007-08-30 14:42:11.870

V

select convert(char, getdate(), 111) as date, convert(char, getdate(), 108) as [time]

==> Date = 2007/08/30

==> Time = 14:44:50

Others datetime format

century century
(yy) (yyyy) standard output format
- - --
- 0 or 100 (*) default mon dd yyyy hh:miAM (or PM)
1 101 USA mm/dd/yy
2 102 ANSI yy.mm.dd
3 103 British/French dd/mm/yy
4 104 German dd.mm.yy
5 105 Italian dd-mm-yy
6 106 - dd mon yy
7 107 - mon dd, yy
8 108 - hh:mmTongue Tieds
- 9 or 109 (*) default mon dd yyyy
milliseconds hh:miTongue Tieds:mmmAM (or PM)
10 110 USA mm-dd-yy
11 111 Japan yy/mm/dd
12 112 ISO yymmdd
- 13 or 113 (*) Europe default dd mon yyyy
milliseconds hh:mmTongue Tieds:mmm(24h)
14 114 - hh:miTongue Tieds:mmm(24h)

(*) the default values (style 0 or 100, 9 or 109, and
13 or 113) always return the century (yyyy)

|||

You have to use Derived Column Transformation of Data Flow.

So:

1. Add a Data Flow task to Control Flow window

2.Add OleDB Source then Derived Column and Finaly Flat File Destination linked with arrow

3. Double Click Derived Column and

write Derived Column Name desired then <add as new column> to Derived Column then in expresion : (DT_DBTIME)YourDataColumnName and Data Type database time[DT_DBTIME]

in second line write Derived Column Name desired then Replace 'YourDataColumnName' and in expression (DT_DBDATE)YourDataColumnName and Data Type databasedate[DT_DBDATE]

|||

I would use this expressions in a derived column:

For Date:

RIGHT("00" + (DT_WSTR,2)MONTH(MyDate),2) + "-" + RIGHT("00" + (DT_WSTR,2)DAY(MyDate),2) + "-" + (DT_WSTR,4)YEAR(MyDate)

For time:

(DT_DBTIME)MyDate