ForEach File Enumerator extension bug?

I set up a basic ForEach enumerator loop and specified files of type *.sql.

In the directory, I had some files I needed to keep but didn't want the package to touch, so I changed the extension to *.sqlo. Much to my dismay, the ForEach loop picked up those files. (Though it did skip the *.xml and *.bat and a few other types...)

Sounds like a bug to me.

That is expected wildcard behaviour. At least, it is what I would expect. To verify I wasn't 'shrooming, I checked out: http://msdn2.microsoft.com/en-us/library/wz42302f.aspx where I found this to put my mind at ease ...

Note

When using the asterisk wildcard character in a searchPattern, such as "*.txt", the matching behavior when the extension is exactly three characters long is different than when the extension is more or less than three characters long. A searchPattern with a file extension of exactly three characters returns files having an extension of three or more characters, where the first three characters match the file extension specified in the searchPattern. A searchPattern with a file extension of one, two, or more than three characters returns only files having extensions of exactly that length that match the file extension specified in the searchPattern. When using the question mark wildcard character, this method returns only files that match the specified file extension. For example, given two files, "file1.txt" and "file1.txtother", in a directory, a search pattern of "file?.txt" returns just the first file, while a search pattern of "file*.txt" returns both files.

Donald

|||

I believe you (after looking at that link). But that is just bizarre and, to me, nonsensical. Especially if one continues to read. This *only* applies to a 3-character extension search pattern - nothing else.

Why oh why would they do such a stupid thing?

This means that there is direct no way to specify give me only the three character file extension files that I've told you to give me.

Were they shrooming when they came up with this?

If I wanted the other extensions I'd ask for "*.txt*".

ForEach Config validation errors

I have a Config file for a Data Flow that is utilizing a Configuration file. I have specified in the Configuration file to capture the Directory and FileSpec for the ForEach Enumeration.

When I open the package, I get the following error:

Warning 1 Warning loading ISOTypeLoss.dtsx: The package path referenced an object that cannot be found: "\Package\Foreach TC03 File.ForEachEnumerator.Properties[Directory]". This occurs when an attempt is made to resolve a package path to an object that cannot be found. i:\visual studio 2005\projects\claimtran\isotypeloss\ISOTypeLoss.dtsx 1 1

Warning 2 Warning loading ISOTypeLoss.dtsx: The package path referenced an object that cannot be found: "\Package\Foreach TC03 File.ForEachEnumerator.Properties[FileSpec]". This occurs when an attempt is made to resolve a package path to an object that cannot be found. i:\visual studio 2005\projects\claimtran\isotypeloss\ISOTypeLoss.dtsx 1 1

I have verified that these two entries are indeed in the Config file and they do point to a valid location for the Directory and File Specification. The package runs fine.

Thanks.

More information:

When I am selecting these two fields during the Configuration file wizard, the final screen that shows the summary of activity displays the following info under the Properties section:

Properties:
<Unknown>.Properties[FileSpec]
<Unknown>.Properties[Directory]
\Package.Connections[LoggingOutput].Properties[ConnectionString]
\Package.Connections[DSSStaging].Properties[ServerName]
\Package.Connections[DSS].Properties[ServerName]
\Package.Connections[Corp1Staging].Properties[ServerName]

The FileSpec and Directory properties are under the ForEach.EnumeratorProperties section.

Is there a known issue pertaining to this?

|||What build are you running?|||

KirkHaselden wrote:

What build are you running?

June CTP|||UPDATE:

We deployed the dtsx to the file system on the server, made changes to the FileSpec parameter in the config file and deployed it, then executed the package pointing to the Config file.

The package still looked for the file as specified in the dtsx XML (package), regardless of what was specified in the config.

To confirm, we renamed the input file to match the FileSpec parameter in the package and it ran successfully, finding and processing the file.

The ForEach parsing for the Enumeration objects appears to have a bug. Can someone let me know if this is a known bug and if it is going to be fixed for the next CTP release?

Thanks.|||

We attempted to recreate the Config file uinder the September CTP and are having the same problems. The Config wizard

Here is what is displayed at the bottom of the dtsConfig file:

File name:
\\MSSQLYukon\etlconfig\ISOTypeLoss.dtsConfig

Properties:
\Package.Connections[ISOTypeLossLog].Properties[ConnectionString]
\Package.Connections[DSSStaging].Properties[ServerName]
\Package.Connections[DSSStaging].Properties[InitialCatalog]
\Package.Connections[DSSStaging].Properties[ConnectionString]
<Unknown>.Properties[FileSpec]
<Unknown>.Properties[Directory]

We still get the error in the Error Window:

Warning 1 Warning loading ISOTypeLoss.dtsx: The package path referenced an object that cannot be found: "\Package\Foreach TC03 File.ForEachEnumerator.Properties[Directory]". This occurs when an attempt is made to resolve a package path to an object that cannot be found. i:\visual studio 2005\projects\claimtran\isotypeloss\ISOTypeLoss.dtsx 1 1

|||You will get this error if your config file contains references to names that don't exist elsewhere in the package. (In this case, it sounds like your config file contains references to "Foreach TC03 File", which doesn't exist in your package.) When you recreated the config file, did you actually delete the previous config file from the file system?|||

Cim Ryan wrote:

You will get this error if your config file contains references to names that don't exist elsewhere in the package. (In this case, it sounds like your config file contains references to "Foreach TC03 File", which doesn't exist in your package.) When you recreated the config file, did you actually delete the previous config file from the file system?

Absolutely. The errors I have highlighted above were copied and pasted from the confirmation screen of completing the Package Configuration Wizard and BEFORE the file is written.

The wizard itself does not recognize the setting before the file is written.

This is a HUGE issue if we are to deploy Yukon! |||

I've only been able to reproduce the error you're encountering by manually editing the config file and corrupting the name of the ForEach Loop. If you'll remove from your package all but the tasks necessary to reproduce the error (make a backup copy first, obviously), I'll take a look at the pared down package if you post it here.

|||

Cim Ryan wrote:

I've only been able to reproduce the error you're encountering by manually editing the config file and corrupting the name of the ForEach Loop. If you'll remove from your package all but the tasks necessary to reproduce the error (make a backup copy first, obviously), I'll take a look at the pared down package if you post it here.

Where in the Wizard should I be selecting the Directory and FileSpec properties? I select them under the <PackageName>/Executables/<ForEachContainerName>/Executables/EnumeratorProperties entry. In there I select "Directory" and "FileSpec".

I tried to post the XML but the Forum logged an error. Can I send the XML to you via email or post an attachment somewhere?|||Yeah, that sounds like the correct place to be specifying the properties. Go ahead and email me the package and config files.|||By the way, these are not errors, they are warnings which are non-critical. There is a property on packages called "SuppressConfigurationWarnings" that will hide these messages. They are there to simply inform you that the configuration is attempting to configure an object that is not there. So you can either remedy the situation or ignore it.
Configuration warnings should not fail the package.
K|||

KirkHaselden wrote:

By the way, these are not errors, they are warnings which are non-critical. There is a property on packages called "SuppressConfigurationWarnings" that will hide these messages. They are there to simply inform you that the configuration is attempting to configure an object that is not there. So you can either remedy the situation or ignore it.
Configuration warnings should not fail the package.
K

Kirk:

The problem is that we need to change settings in the config file to migrate the packages through Model, QA and to Production. As I posted above, I can change the settings to point to a different Directory but because the package cannot configure the setting it always uses the setting that was set when the package was designed, which is the Test directory. The only way to change this is to open the package, change the attributes during design and save them. But then it points to the Production directory, etc.

Changing the SuppressconfiurationWarnings setting will indeed hide the warning but it will not change the setting at execution time - which is the reason for having these properties in the config file in the first place.|||

Cim Ryan wrote:

Yeah, that sounds like the correct place to be specifying the properties. Go ahead and email me the package and config files.

Done. As stated, I am running September CTP.|||

I wasn't able to reproduce this failure, and Raul can no longer do so, either.

ForEach Config validation errors

I have a Config file for a Data Flow that is utilizing a Configuration file. I have specified in the Configuration file to capture the Directory and FileSpec for the ForEach Enumeration.

When I open the package, I get the following error:

Warning 1 Warning loading ISOTypeLoss.dtsx: The package path referenced an object that cannot be found: "\Package\Foreach TC03 File.ForEachEnumerator.Properties[Directory]". This occurs when an attempt is made to resolve a package path to an object that cannot be found. i:\visual studio 2005\projects\claimtran\isotypeloss\ISOTypeLoss.dtsx 1 1

Warning 2 Warning loading ISOTypeLoss.dtsx: The package path referenced an object that cannot be found: "\Package\Foreach TC03 File.ForEachEnumerator.Properties[FileSpec]". This occurs when an attempt is made to resolve a package path to an object that cannot be found. i:\visual studio 2005\projects\claimtran\isotypeloss\ISOTypeLoss.dtsx 1 1

I have verified that these two entries are indeed in the Config file and they do point to a valid location for the Directory and File Specification. The package runs fine.

Thanks.

More information:

When I am selecting these two fields during the Configuration file wizard, the final screen that shows the summary of activity displays the following info under the Properties section:

Properties:
<Unknown>.Properties[FileSpec]
<Unknown>.Properties[Directory]
\Package.Connections[LoggingOutput].Properties[ConnectionString]
\Package.Connections[DSSStaging].Properties[ServerName]
\Package.Connections[DSS].Properties[ServerName]
\Package.Connections[Corp1Staging].Properties[ServerName]

The FileSpec and Directory properties are under the ForEach.EnumeratorProperties section.

Is there a known issue pertaining to this?

|||What build are you running?|||

KirkHaselden wrote:

What build are you running?

June CTP|||UPDATE:

We deployed the dtsx to the file system on the server, made changes to the FileSpec parameter in the config file and deployed it, then executed the package pointing to the Config file.

The package still looked for the file as specified in the dtsx XML (package), regardless of what was specified in the config.

To confirm, we renamed the input file to match the FileSpec parameter in the package and it ran successfully, finding and processing the file.

The ForEach parsing for the Enumeration objects appears to have a bug. Can someone let me know if this is a known bug and if it is going to be fixed for the next CTP release?

Thanks.|||

We attempted to recreate the Config file uinder the September CTP and are having the same problems. The Config wizard

Here is what is displayed at the bottom of the dtsConfig file:

File name:
\\MSSQLYukon\etlconfig\ISOTypeLoss.dtsConfig

Properties:
\Package.Connections[ISOTypeLossLog].Properties[ConnectionString]
\Package.Connections[DSSStaging].Properties[ServerName]
\Package.Connections[DSSStaging].Properties[InitialCatalog]
\Package.Connections[DSSStaging].Properties[ConnectionString]
<Unknown>.Properties[FileSpec]
<Unknown>.Properties[Directory]

We still get the error in the Error Window:

Warning 1 Warning loading ISOTypeLoss.dtsx: The package path referenced an object that cannot be found: "\Package\Foreach TC03 File.ForEachEnumerator.Properties[Directory]". This occurs when an attempt is made to resolve a package path to an object that cannot be found. i:\visual studio 2005\projects\claimtran\isotypeloss\ISOTypeLoss.dtsx 1 1

|||You will get this error if your config file contains references to names that don't exist elsewhere in the package. (In this case, it sounds like your config file contains references to "Foreach TC03 File", which doesn't exist in your package.) When you recreated the config file, did you actually delete the previous config file from the file system?|||

Cim Ryan wrote:

You will get this error if your config file contains references to names that don't exist elsewhere in the package. (In this case, it sounds like your config file contains references to "Foreach TC03 File", which doesn't exist in your package.) When you recreated the config file, did you actually delete the previous config file from the file system?

Absolutely. The errors I have highlighted above were copied and pasted from the confirmation screen of completing the Package Configuration Wizard and BEFORE the file is written.

The wizard itself does not recognize the setting before the file is written.

This is a HUGE issue if we are to deploy Yukon! |||

I've only been able to reproduce the error you're encountering by manually editing the config file and corrupting the name of the ForEach Loop. If you'll remove from your package all but the tasks necessary to reproduce the error (make a backup copy first, obviously), I'll take a look at the pared down package if you post it here.

|||

Cim Ryan wrote:

I've only been able to reproduce the error you're encountering by manually editing the config file and corrupting the name of the ForEach Loop. If you'll remove from your package all but the tasks necessary to reproduce the error (make a backup copy first, obviously), I'll take a look at the pared down package if you post it here.

Where in the Wizard should I be selecting the Directory and FileSpec properties? I select them under the <PackageName>/Executables/<ForEachContainerName>/Executables/EnumeratorProperties entry. In there I select "Directory" and "FileSpec".

I tried to post the XML but the Forum logged an error. Can I send the XML to you via email or post an attachment somewhere?|||Yeah, that sounds like the correct place to be specifying the properties. Go ahead and email me the package and config files.|||By the way, these are not errors, they are warnings which are non-critical. There is a property on packages called "SuppressConfigurationWarnings" that will hide these messages. They are there to simply inform you that the configuration is attempting to configure an object that is not there. So you can either remedy the situation or ignore it.
Configuration warnings should not fail the package.
K|||

KirkHaselden wrote:

By the way, these are not errors, they are warnings which are non-critical. There is a property on packages called "SuppressConfigurationWarnings" that will hide these messages. They are there to simply inform you that the configuration is attempting to configure an object that is not there. So you can either remedy the situation or ignore it.
Configuration warnings should not fail the package.
K

Kirk:

The problem is that we need to change settings in the config file to migrate the packages through Model, QA and to Production. As I posted above, I can change the settings to point to a different Directory but because the package cannot configure the setting it always uses the setting that was set when the package was designed, which is the Test directory. The only way to change this is to open the package, change the attributes during design and save them. But then it points to the Production directory, etc.

Changing the SuppressconfiurationWarnings setting will indeed hide the warning but it will not change the setting at execution time - which is the reason for having these properties in the config file in the first place.|||

Cim Ryan wrote:

Yeah, that sounds like the correct place to be specifying the properties. Go ahead and email me the package and config files.

Done. As stated, I am running September CTP.|||

I wasn't able to reproduce this failure, and Raul can no longer do so, either.

FOR XML performance question

I am currently rewriting a data access component to make use of the FOR XML
SQL statement to return XML data as an ADO stream from a specified source.
The older current component requests this data using an ADO recordset and
then manually converts this to XML.
I have run several performance tests comparing the 2 and on narrow and
medium width tables I have found that the performance gain is massive (appro
x
80% gain). However, when I run the 2 on very wide tables, ones which contai
n
text/ntext columns, FOR XML only performs about 10% better pulling back 1 ro
w
but pulling back 20 rows it becomes over twice as slow as the older componen
t.
Can anyone suggest why? Or even better, any methods/tips that could improve
performance in this instance?
Thanks in advance.I'm cross posting this to microsoft.public.sqlserver.xml, a more appropriate
forum for this question.
--
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Lee" <Lee@.discussions.microsoft.com> wrote in message
news:F314E284-6E34-403C-94C1-B307682B1418@.microsoft.com...
>I am currently rewriting a data access component to make use of the FOR XML
> SQL statement to return XML data as an ADO stream from a specified
> source.
> The older current component requests this data using an ADO recordset and
> then manually converts this to XML.
> I have run several performance tests comparing the 2 and on narrow and
> medium width tables I have found that the performance gain is massive
> (approx
> 80% gain). However, when I run the 2 on very wide tables, ones which
> contain
> text/ntext columns, FOR XML only performs about 10% better pulling back 1
> row
> but pulling back 20 rows it becomes over twice as slow as the older
> component.
> Can anyone suggest why? Or even better, any methods/tips that could
> improve
> performance in this instance?
> Thanks in advance.|||I am currently rewriting a data access component to make use of the FOR XML
SQL statement to return XML data as an ADO stream from a specified source.
The older current component requests this data using an ADO recordset and
then manually converts this to XML.
I have run several performance tests comparing the 2 and on narrow and
medium width tables I have found that the performance gain is massive (appro
x
80% gain). However, when I run the 2 on very wide tables, ones which contai
n
text/ntext columns, FOR XML only performs about 10% better pulling back 1 ro
w
but pulling back 20 rows it becomes over twice as slow as the older componen
t.
Can anyone suggest why? Or even better, any methods/tips that could improve
performance in this instance?
Thanks in advance.
----
In addition to the above I have done some further investigation. On a query
which returns the top row from a table the FOR XML method performed 83.5%
faster than the recordset version. However, when I run a where query which
I
know returns a single row the FOR XML method performance plunges and is
actually 6% slower than the recordset version.
Is SQLXML just one of those things which seems like a great idea but has no
real practical use in an enterprise environment? I find it very frustrating
that its performance is superb in some situations but is so awful in others.
Is it a work in progress?
That said, are there any resources which discuss various ways to pull data
from SQL server 2000 as(and convert to) XML format? Surely there is a bette
r
way than using a ADO recordset as described above?
Thanks.|||Hi Lee
This is hard to answer without having more specifics.
How does your FOR XML query look like? How does it compare to the previous
query, what indices do yo have on it?
Etc.
Best regards
Michael
"Lee" <Lee@.discussions.microsoft.com> wrote in message
news:95F97E2B-CCE3-415F-AEBE-25E7B499825E@.microsoft.com...
>I am currently rewriting a data access component to make use of the FOR XML
> SQL statement to return XML data as an ADO stream from a specified
> source.
> The older current component requests this data using an ADO recordset and
> then manually converts this to XML.
> I have run several performance tests comparing the 2 and on narrow and
> medium width tables I have found that the performance gain is massive
> (approx
> 80% gain). However, when I run the 2 on very wide tables, ones which
> contain
> text/ntext columns, FOR XML only performs about 10% better pulling back 1
> row
> but pulling back 20 rows it becomes over twice as slow as the older
> component.
> Can anyone suggest why? Or even better, any methods/tips that could
> improve
> performance in this instance?
> Thanks in advance.
> ----
> In addition to the above I have done some further investigation. On a
> query
> which returns the top row from a table the FOR XML method performed 83.5%
> faster than the recordset version. However, when I run a where query
> which I
> know returns a single row the FOR XML method performance plunges and is
> actually 6% slower than the recordset version.
> Is SQLXML just one of those things which seems like a great idea but has
> no
> real practical use in an enterprise environment? I find it very
> frustrating
> that its performance is superb in some situations but is so awful in
> others.
> Is it a work in progress?
> That said, are there any resources which discuss various ways to pull data
> from SQL server 2000 as(and convert to) XML format? Surely there is a
> better
> way than using a ADO recordset as described above?
> Thanks.|||The query is a simple :-
SELECT stuff
FROM table
WHERE condition (optional)
FOR XML RAW
There is a single index on the primary key of the table and that was the
field I did my where clause on as described in my above posts.
"Michael Rys [MSFT]" wrote:

> Hi Lee
> This is hard to answer without having more specifics.
> How does your FOR XML query look like? How does it compare to the previous
> query, what indices do yo have on it?
> Etc.
> Best regards
> Michael
> "Lee" <Lee@.discussions.microsoft.com> wrote in message
> news:95F97E2B-CCE3-415F-AEBE-25E7B499825E@.microsoft.com...
>
>