Force to complete query, ignore errors

Hi,

I have a big table and want to make a plausibility check of it′s data.

Problem is, that my query stops, if there is an unexpected datatype in one of the rows. But that is it, what i want to filter out of my table with that query and save the result as new correct table.

How can i write a parameter to my query SQL Code, that if a error occurs, the querry resumes and the error line will not displayed in my final querry overview?

In my books and on the net, i don′t found something to this theme ;-(.

Thx in advance.

You should provide additional info
- how is the table outline ?
- which is the unexpected data that breaks your code ?
- which sql server version are you working with ?
- it's a pure t-sql approach or is a mixed ado.net / sql approach ?
- can you post a short version of the t-sql code here ?|||- how is the table outline ?
The table has 30 columns with differnt data's in there. The datatyps are all nvarchar(50) after the flatfile txt Import. But there are some date, text, int and float datats, in there. I will bringt the correct datatyp to the columns later, because i have 80 of such tables, and before i must correct them form wrong lines and merge them after that.

- which is the unexpected data that breaks your code ?

Very very much possible things. A date where a int is expected. A float where a int is expected and and. It′s because the txt-Files i got have many data errors and wrong moved lines in there and i must correct them now.

- which sql server version are you working with ?

SQL Server 2005

- it's a pure t-sql approach or is a mixed ado.net / sql approach ?

I don′t know, but i don′t think ado.net.

- can you post a short version of the t-sql code here ?

[Code]
SELECT TOP (100) PERCENT _Ti, _Pl, _Ve, [_Datum_Time], _Ere, _Sg, _Sm,
FROM dbo.d6020900_ges
WHERE (_Ti < 1) OR
(_Ti > 1000) AND (_Pl < 0) OR
(_Pl > 15) AND (_Ve < 0) OR
(_Ve > 10000) AND (_Ere < - 1) OR
(_Ere > 10) AND (_Sg < N'0') OR
(_Sg > N'356') AND (_Sm < N'0') OR
(_Sm > N'255')
ORDER BY _Ti, _Pl, _Ve, [_Datum_Time]
[\Code]

|||There are different approaches depending on which is the operation you have to do...

First of all don't expect that is a simple task... I don't think you will be able to build a query like the one you posted... there are a lot of implicit conversions that simply cannot work in your case... there's no way to tell SQL to "skip" some row when a conversion fails... you have to build a little "engine" that anayze data column by column.

If you have to correct data in a table, this happens only once, so you can launch a long running query, maybe using a cursor and evaluating on a per row basis...
if this is the case you may also want to use a CLR stored proc... in that case you may fetch your data into a dataset and use standard manipulation/conversion using your favorite language (C#, VB.NET). In that case I think that the use of a CLR SP is fully justified from the improved flexibility in analyzing data, after all string manipulation with the framework is a lot more powerful that T-SQL.

If you have to repeadetly query the data to build a report or something like that... well this is a nightmare... I strongly suggest you to convert your data into a table that has the required data types (integers, datetime and so on), else you may build a bunch of conversion functions and/or a view made up by functions (or computed fields) that may be used to access the data... but, as you probably have already thought, this not a good performance solution.. it would be better to apply those functions only once to migrate your data into a brand new ad-hoc table.

Anyway let me know if you need further info.|||Thx so far.

My biggest problem are the much wrong lines in my Table, so converting in SQL fails and importing the basis txt-flatfile to a table with the correct datatypes fails also.

Is there a way to import only the rows, that are correct and dont stop at the wrong?|||You need to process you data row by row. This can be done
- using a cursor if you want to operate with a pure T-SQL solution
- using an external application (or a CLR stored proc) if you want more power/flexibility
I don't know any other way

You may also add a signature bit to your row that marks the row as dirty so you can filter it out... but again this requires that you process the table line by line.

in pseudo-code this may look like this

OPEN CURSOR
FETCH NEXT DATA INTO FIELD1, FIELD2, ...
WHILE (@.@.FETCH_STATUS = 0) BEGIN
IF DIRTY_DATE(FIELD1) OR DIRTY_INT(FIELD2) OR ... BEGIN
... DO SOMETHING HERE ...
END
FETCH NEXT DATA INTO FIELD1, FIELD2, ...
END

Just a question: why don't you avoid inserting bad rows when you import the txt file ?|||Hm, what u mean with avoid inserting bad rows? The bad rows are allready in my base txt-Files. But they are too large, to check and delete there every single line with the hand.

I don′t know how i can say on the import assistent, that bad rows will not be importet. At my try's the import stops, if a bad row is detected, an i musst correct the line in the txt-file with my own hands, delete the table in the db and then start again the import.

Is there a better way?|||> Hm, what u mean with avoid inserting bad rows?

You have an application that imports the text file into the system... correct ? well in that case the validation must be performed from that application. the data are imported only after validation ha occurred, so you won't have any problem in the db.

> Is there a better way?

I will be explicit, hope you won't offend:
1. who is the mad person that thought such a procedure ? how is possible that your "file-producer" is not able to write a decent file with no errors ? What I would do ? completely reject any file that contains errors.
2. who have designed the table with the nvarchar data only ? hope that you agree that is really a stupid thing... it's like you were trying to program an application with no data types... only pointers and bytes... that's prehistory.
|||O.K. thx, but the produced basis txt Files are corrupt and i have filtered many things before with VBA in excel, but to filter everything, vba and excel are to slow for my masses on files. The basis txt files are so, like I have got them, i can't get new ones or better ones, i must live with them.|||
> i must live with them.

ognuno ha le sue sfighe ...

For XML Problem with IIS6 and W2k3

Did anyone ever resolve this? I am having the exact same issue...returning
results from a FOR XML procedure to an ado stream object stops working every
several days.
"ajsmith02" wrote:

> I have this function that worked like a charm under IIS5 and W2K. You pas
s a
> sql string that has for xml auto or a stored produre that has for xml auto
in
> it. Under IIS6 and W2K3 it stops working after a couple of days with no
> rhyme or reason. No error log either. We applied all the service packs
> including sqlxml sp3. What is wrong? Thanks.
> Here is the code:
> function getSQLXML(byval sqlString)
> dim adoConn
> dim adoCmd
> dim adoStreamQuery
> set adoConn = vbsqlconnection 'located in sharedfunctions.asp
> adoConn.CommandTimeout = 300
> set adoStreamQuery = Server.CreateObject("ADODB.Stream")
> set adoCmd = Server.CreateObject("ADODB.Command")'
> adoCmd.ActiveConnection = adoConn
> adoCmd.CommandTimeout = 300
> adoConn.CursorLocation = adUseClient
> dim sQuery
> sQuery = "<recordset xmlns:sql='urn:schemas-microsoft-com:xml-sql'>"
> sQuery = sQuery + "<sql:query>"+sqlString+"</sql:query>"
> sQuery = sQuery + "</recordset>"
> adoStreamQuery.Open 'Open the command stream so it may be written to
> adoStreamQuery.WriteText sQuery, adWriteChar 'Set the input command
> stream's text with the query string
> adoStreamQuery.Position = 0 'Reset the position in the stream, otherwis
e
> it will be at EOS
> adoCmd.Dialect = "{5D531CB2-E6Ed-11D2-B252-00C04F681B71}" 'Set the
> dialect for the command stream to be a SQL query.
> adoCmd.CommandStream = adoStreamQuery 'Set the command object's comma
nd
> to the input stream set above
> dim outStrm
> set outStrm = Server.CreateObject("ADODB.Stream") 'Create the output
> stream
> outStrm.Open
> adoCmd.Properties("Output Stream").Value = outStrm 'Set command's output
> stream to the output stream just opened
> adoCmd.Execute , , adExecuteStream
> 'Response.Write(outStrm.ReadText)
> adoCmd.ActiveConnection = nothing
> adoConn.Close
> set adoConn = nothing
> getSQLXML = outStrm.ReadText
> end function
> P.S. Goorbeeman in the group microsoft.public.sqlserver.server has the sam
e
> problemNobody helped me. It turned out to be a blessing in disguise. I wound up
creating a component using .Net to pass through the "For XML" sql statements
and return the xml in for of a text stream. After I created the component I
Com Wrapper (using .NET) so that my old asp page to use it.
"ashort" wrote:
> Did anyone ever resolve this? I am having the exact same issue...returnin
g
> results from a FOR XML procedure to an ado stream object stops working eve
ry
> several days.
> "ajsmith02" wrote:
>|||Folks,
Has ANYONE solved this problem? I am getting "Catastrophic failure" message
s and Err.Number = -2147418113 from my ASP pages, consistently.
Thanks in advance...
- Tom

quote:

---

Originally posted by ashort
[B]Did anyone ever resolve this? I am having the exact same issue...returning
results from a FOR XML procedure to an ado stream object stops working every
several days.
"ajsmith02" wrote:
> I have this function that worked like a charm under IIS5 and W2K. You pas
s a
> sql string that has for xml auto or a stored produre that has for xml auto
in
> it. Under IIS6 and W2K3 it stops working after a couple of days with no
> rhyme or reason.

---

F|||Hi,
Could anyone reproduce the problem with a generic ASP page against e.g.
Northwind database? If so, please send me the code if possible. I have the
following ASP page and run it for days on Windows Server 2003 and SQLXML3
SP3, but didn't see it hanging using a stress app.
Also, Windows Server 2003 SP1 can be downloaded, so if someone could give it
a try, it would be great.
<!--#include file="common.inc"-->
<% Response.ContentType = "text/xml" %>
<object id="conn" progid="ADODB.Connection" runat="Server"></object>
<%
Dim strSQL, getSQLXML, strSQLXML, objCmd
dim outStrm
set outStrm = Server.CreateObject("ADODB.Stream") 'Create the output
stream
outStrm.Open
strSQL = "<sql:query>select * from orders for xml auto</sql:query>"
strSQLXML = "<?xml version=""1.0"" ?><root
xmlns:sql='urn:schemas-microsoft-com:xml-sql'>" & strSQL & "</root>"
conn.Open strConnNew
Set objCmd = Server.CreateObject("ADODB.Command")
objCmd.ActiveConnection = conn
objCmd.CommandText = strSQLXML
objCmd.Dialect = "{5d531cb2-e6ed-11d2-b252-00c04f681b71}"
objCmd.Properties("Output Stream").Value = outStrm
objCmd.Execute , , 1024
outStrm.Position = 0
outStrm.Charset = "utf-8"
getSQLXML = outStrm.ReadText(-1)
outStrm.Close
Response.Write("<root/>")
%>
thx,
-kuen
This posting is provided "AS IS" with no warranties, and confers no rights.
"TomKelleher" wrote:

> Folks,
> Has ANYONE solved this problem? I am getting "Catastrophic failure"
> messages and Err.Number = -2147418113 from my ASP pages, consistently.
>
> Thanks in advance...
> - Tom
> ashort wrote:
> F
>
> --
> TomKelleher
> ---
> Posted via http://www.codecomments.com
> ---
>|||Thanks for the explanation. I am checking with the MDAC team to see if they
are aware of the issue.
Thanks
Michael
<spamgone@.cox.net> wrote in message
news:1114110540.157389.251100@.g14g2000cwa.googlegroups.com...
>I have started experiencing the same problem when my company switched
> to the Windows 2003 servers. Since the switch, our asp web pages that
> contain the adodb.stream objects will randomly lockup until the
> application pool is recycled in IIS. Then everything will run smoothly
> for a few days. All ASP pages that don't contain the adodb.stream
> object will continue to work as normal, when the lockup occurs. No
> errors are generated in the logs and the web server and database server
> doesn't show any signs of heavy memory or CPU usage. You can even
> run the "FOR XML AUTO" stored procedures in query analyzer without
> any errors during this time. To try and prevent the problem we setup
> the application pools to be recycled nightly, but that doesn't appear
> to make any difference. We have Microsoft SQL Server on a different
> machine then our web server, but I doubt that makes a difference. The
> offending ASP pages were working perfectly fine without any issues when
> the server was still Windows 2000 Server.
> Here is the basics of the offending ASP pages:
> <%@. Language=VBScript %>
> <%Option Explicit%>
> <%
> Dim objCommand,objXML,objStream,objRoot
>
> Set objCommand = Server.CreateObject("ADODB.Command")
> Set objXML = Server.CreateObject("MSXML2.DomDocument")
> Set objStream = Server.CreateObject("ADODB.Stream")
>
> objStream.Open
> With objCommand
> .ActiveConnection = "Provider=SQLOLEDB; Data Source=ExampleServer;
> Network Library=DBMSSOCN; Initial Catalog=ExampleDB; User Id=Example;
> Password=Example"
> .CommandType = adCmdStoredProc
> .CommandText = "SP_Example"
> .Properties("Output Stream") = objStream
> .Execute ,, adExecuteStream
> End With
> objXML.loadXML("<root>" & objStream.ReadText & "</root>")
> Set objRoot = objXML.documentElement
> %>
> I have not included the rest of the code that displays the returned
> data because the page does not appear to get past the Execute
> statement.
> The stored procedure is basically:
> CREATE PROCEDURE SP_Example
> AS
> SELECT ExampleID, ExampleName from ExampleTable
> FOR XML AUTO
> GO
> Any suggestions, beyond recoding every web page, would be greatly
> appreciated.
> Thanks.
> Paul
>|||Any result from the MDAC team? I am fighting the same issue, and would be
very interested in a solution or workaround.
"Michael Rys [MSFT]" wrote:

> Thanks for the explanation. I am checking with the MDAC team to see if the
y
> are aware of the issue.
> Thanks
> Michael
> <spamgone@.cox.net> wrote in message
> news:1114110540.157389.251100@.g14g2000cwa.googlegroups.com...
>
>|||MDAC team hasn't seen the problem you describe in our internal tests.
Does this page lockup mean that the thread executing the script hangs?
If so is it possbile to get the stack trace and the version information for
the modules involved?
This posting is provided "AS IS" with no warranties, and confers no rights.
"Bob Hobnob" wrote:
> Any result from the MDAC team? I am fighting the same issue, and would be
> very interested in a solution or workaround.
> "Michael Rys [MSFT]" wrote:
>|||Below I have copied my post from the data.ado group - it contains my ASP cod
e
and the IISState log for the problem thread. I can provide module versions,
if you wouldn't mind telling me which dll's to examine. I did just install
the MDAC 2.8 upgrade, but the behavior was the same before as after. I'm no
t
sure how to provide a stack trace, but I'll see if I can Google something on
it. I'd be happy to provide anything I can to help solve this issue.
//bob
--snip--
Like many others, I am struggling with a problem in Win2K3 and IIS6 using an
ADO Stream object and an SQL FOR XML stored procedure. Let me preface by
saying that the ASP code has worked flawlessly on Win2K and IIS 5 for over a
year. Now on Win2K3, the page will work for a while, then it becomes
unresponsive. It returns no error, it just hangs the connection so that no
other site page will respond until the browser is closed and re-opened.
Sometimes recycling the app pool will clear it up temporarily, sometimes I
have to restart IIS. Page will work fine for a few hours or even days, then
it stops responding. CPU and memory utilization seem normal in TaskMan.
First, here's the ASP code:
<%
dim cn
set cn = server.CreateObject("ADODB.Connection")
cn.open MM_iiWeb_STRING
dim result
result = getSQLXML(cn, "exec proc_getLinkCatXML", "linkpage.xsl", ".")
response.Write result
if cn.state=1 then cn.close()
set cn = nothing
%>
<%
function getSQLXML(byref cn, byval sql, byval xslfile, byval basepath)
dim cmd
dim objOutStream
Set objOutStream = Server.CreateObject("ADODB.Stream")
objOutStream.open
Set cmd = Server.CreateObject("ADODB.Command")
cmd.ActiveConnection = cn
cmd.CommandText = sql
cmd.Properties("Base Path").Value = Server.MapPath(basepath)
cmd.Properties("XML Root") = "root"
cmd.Properties("XSL") = xslfile
cmd.Properties("Output Stream") = objOutStream
cmd.Execute , , adExecuteStream
getSQLXML = objOutStream.ReadText
objOutStream.Close
set objOutStream = nothing
set cmd = nothing
end function
%>
When it hangs, here is IISState log info for the thread:
Thread ID: 15
System Thread ID: bc0
Kernel Time: 0:0:0.109
User Time: 0:0:0.968
Thread Type: ASP
Executing Page: C:\INETPUB\WWWROOT\IIWEB\TEMPLATES\LINKS
.ASP
# ChildEBP RetAddr
00 025de3d8 77f4262b SharedUserData!SystemCallStub+0x4
01 025de3dc 77e418ea ntdll!NtDelayExecution+0xc
02 025de444 77e416ee kernel32!SleepEx+0x68
03 025de450 0439c4f9 kernel32!Sleep+0xb
04 025de464 04382a20 mo15!CQuery::Cancel+0x50
05 025de49c 0437d84d mo15!CCommand::Cancel+0x58
06 025de4ac 0437516f mo15!CCommand::Term+0xd
07 025de4c8 04373951 mo15!CStdSymbiontObject::InternalRele
ase+0x6c
08 025de4d8 7710736a mo15!ATL::CComObject<CRecordset>::Release+0x11
09 025de4e8 7346384b OLEAUT32!VariantClear+0xad
0a 025de4fc 734641cf vbscript!VAR::Clear+0xab
0b 025de50c 7346416a vbscript!CScriptRuntime::Cleanup+0x59
0c 025de84c 73465184 vbscript!CScriptRuntime::Run+0x2ccc
0d 025de504 80020102 vbscript!CScriptRuntime::Run+0x99
WARNING: Frame IP not in any known module. Following frames may be wrong.
0e 025de504 80020102 0x80020102
0f 00000000 00000000 0x80020102
I have seen the same problem posted several times in various newsgroups and
message boards for the past 8 months or so, but I have seen no explaination,
solution, or workaround offered. I would appreciate any insights.
//bob
--end snip--
"Anton Klimov [MS]" wrote:

> MDAC team hasn't seen the problem you describe in our internal tests.
> Does this page lockup mean that the thread executing the script hangs?
> If so is it possbile to get the stack trace and the version information fo
r
> the modules involved?
> --
> This posting is provided "AS IS" with no warranties, and confers no rights
.
>|||The stack you provided shows a problem with releasing a pointer to a rowset.
However the code fragment you posted does not have any rowsets defined,
unless I'm missing something.
You should see where you might have recordsets created with asynchronous
execution.
--
This posting is provided "AS IS" with no warranties, and confers no rights.
"Bob Hobnob" wrote:

> Below I have copied my post from the data.ado group - it contains my ASP c
ode
> and the IISState log for the problem thread. I can provide module version
s,
> if you wouldn't mind telling me which dll's to examine. I did just instal
l
> the MDAC 2.8 upgrade, but the behavior was the same before as after. I'm
not
> sure how to provide a stack trace, but I'll see if I can Google something
on
> it. I'd be happy to provide anything I can to help solve this issue.
> //bob
> --snip--
> Like many others, I am struggling with a problem in Win2K3 and IIS6 using
an
> ADO Stream object and an SQL FOR XML stored procedure. Let me preface by
> saying that the ASP code has worked flawlessly on Win2K and IIS 5 for over
a
> year. Now on Win2K3, the page will work for a while, then it becomes
> unresponsive. It returns no error, it just hangs the connection so that n
o
> other site page will respond until the browser is closed and re-opened.
> Sometimes recycling the app pool will clear it up temporarily, sometimes I
> have to restart IIS. Page will work fine for a few hours or even days, th
en
> it stops responding. CPU and memory utilization seem normal in TaskMan.
> First, here's the ASP code:
> <%
> dim cn
> set cn = server.CreateObject("ADODB.Connection")
> cn.open MM_iiWeb_STRING
> dim result
> result = getSQLXML(cn, "exec proc_getLinkCatXML", "linkpage.xsl", ".")
> response.Write result
> if cn.state=1 then cn.close()
> set cn = nothing
> %>
> <%
> function getSQLXML(byref cn, byval sql, byval xslfile, byval basepath)
> dim cmd
> dim objOutStream
> Set objOutStream = Server.CreateObject("ADODB.Stream")
> objOutStream.open
> Set cmd = Server.CreateObject("ADODB.Command")
> cmd.ActiveConnection = cn
> cmd.CommandText = sql
> cmd.Properties("Base Path").Value = Server.MapPath(basepath)
> cmd.Properties("XML Root") = "root"
> cmd.Properties("XSL") = xslfile
> cmd.Properties("Output Stream") = objOutStream
> cmd.Execute , , adExecuteStream
> getSQLXML = objOutStream.ReadText
> objOutStream.Close
> set objOutStream = nothing
> set cmd = nothing
> end function
> %>
> When it hangs, here is IISState log info for the thread:
> Thread ID: 15
> System Thread ID: bc0
> Kernel Time: 0:0:0.109
> User Time: 0:0:0.968
> Thread Type: ASP
> Executing Page: C:\INETPUB\WWWROOT\IIWEB\TEMPLATES\LINKS
.ASP
> # ChildEBP RetAddr
> 00 025de3d8 77f4262b SharedUserData!SystemCallStub+0x4
> 01 025de3dc 77e418ea ntdll!NtDelayExecution+0xc
> 02 025de444 77e416ee kernel32!SleepEx+0x68
> 03 025de450 0439c4f9 kernel32!Sleep+0xb
> 04 025de464 04382a20 mo15!CQuery::Cancel+0x50
> 05 025de49c 0437d84d mo15!CCommand::Cancel+0x58
> 06 025de4ac 0437516f mo15!CCommand::Term+0xd
> 07 025de4c8 04373951 mo15!CStdSymbiontObject::InternalRele
ase+0x6c
> 08 025de4d8 7710736a mo15!ATL::CComObject<CRecordset>::Release+0x11
> 09 025de4e8 7346384b OLEAUT32!VariantClear+0xad
> 0a 025de4fc 734641cf vbscript!VAR::Clear+0xab
> 0b 025de50c 7346416a vbscript!CScriptRuntime::Cleanup+0x59
> 0c 025de84c 73465184 vbscript!CScriptRuntime::Run+0x2ccc
> 0d 025de504 80020102 vbscript!CScriptRuntime::Run+0x99
> WARNING: Frame IP not in any known module. Following frames may be wrong.
> 0e 025de504 80020102 0x80020102
> 0f 00000000 00000000 0x80020102
> I have seen the same problem posted several times in various newsgroups an
d
> message boards for the past 8 months or so, but I have seen no explainatio
n,
> solution, or workaround offered. I would appreciate any insights.
> //bob
> --end snip--
> "Anton Klimov [MS]" wrote:
>
>|||The code I posted is the complete content of the "links.asp" problem page.
This page is called via an ASP Server.Execute command inside another page.
This "parent" page does have 2 database routines of it's own - both use the
ADODB.Connection "Execute" method to run stored procedures that return
recordsets into variables (no recordset objects explicitly created). As far
as I know, the default "Execute" behavior is synchronous, yes? I certainly
am not specifying the asynchronous option. After the recordsets are
returned, I explicitly close and "nothing" the recordsets, then close and
"nothing" the connection object. This code is all supposed to run and
complete before the "links.asp" page gets executed.
Does any of this info raise a red flag?
//bob
"Anton Klimov [MS]" wrote:

> The stack you provided shows a problem with releasing a pointer to a rowse
t.
> However the code fragment you posted does not have any rowsets defined,
> unless I'm missing something.
> You should see where you might have recordsets created with asynchronous
> execution.
> --
> This posting is provided "AS IS" with no warranties, and confers no rights
.
>

FOR XML output results

Hi,

I've a problem with the result of a FOR XML EXPLICIT query. the output is ok but it suddenly stops. it looks like there is a limit on characters or something.

The output is:

<b100><b110>2141287</b110><b120>GB</b120><b130>A1 </b130><b140>1984-12-12T00:00:00</b140></b100><b200><b210>08408789</b210><b210a>1984 1234</b210a><b211>A</b211><b220>1984-04-05T00:00:00</b220><b250>Kat</b250><b260>Kat</b260></b200><b300><b310>05244683</b3 --and here it stops!!

Does anyone know how to solve this problem?

Greetings,
Renehttp://www.sqlteam.com/forums/topic.asp?TOPIC_ID=25662

macka.