Gotcha, thanks! Seeing how very slow the debugging was, I implemented
it differently, and only a few grey hairs later it seems to work
fine:-)
1) Set the SQL insert statement(s) as an expression string in a
variable with global scope, but with EvaluateAsExpression = FALSE.
This is the only place I need to set the SQL statement(s):
OnPostExecute_Sql = "INSERT INTO ..." + @[User::InsertRow] + ...
OnError_Sql = "INSERT INTO ..." + @[System::ErrorCode] + ...
2) Create a variable with global scope to hold a list of the
counting variables that needs resetting to 0 after each event:
Log_Variables_To_Reset = User::Delete_Count,User::Insert_Count...
3) In the parallel containers, create local row count variables
Insert_Count..., i.e. one set of variables for each container.
This keeps the containers separate when they're writing to
variables with otherwise the same name.
4) Create an event handler in one of the parallel containers, i.e.
with scope limited to one of the parallel containers, it has:
a) A local variable in the event handler with EvaluateAsExpression
= TRUE
User::OnPostExecute_Sql_Local
b) "Expand OnPostExecute_Sql" script with a single line (this is the
'trick' that does an extra EVALUATE() on the SQL expression string,
I couldn't find another way of doing it, but this works fine.)
Dts.Variables("User::OnPostExecute_Sql_Local").Expression = _
Dts.Variables("User::OnPostExecute_Sql").Value.ToString()
c) The Execute SQL task has a property expression with
SqlStatementSource = @[User::OnPostExecute_Sql_Local]
d) A script task to reset the appropriate variables:
Dim Vars As String = Dts.Variables("User::Log_Variables_To_Reset") _
.Value.ToString()
Dim VarArr() As String = Split(Vars, ",")
For i As Integer = 0 To VarArr.Length - 1
Dts.Variables(VarArr(i)).Value = 0
Next
5) Now, for any new parallel container, copy&paste the container and
the two event handlers, and create just the local variable
OnPostExecute_Sql_Local with EvaluateAsExpression=TRUE in the event handler,
and you're done - no need to edit any SQL or expressions in the event
handlers, and the containers can run and log in parallel.
6) To change the SQL insert statement across all parallel containers,
add additional variables to log etc, simply edit
OnPostExecute_Sql, OnError_Sql, Log_Variables_To_Reset
in one place in the global scope.
One gotcha: Enabling precompiled scripts in the event handlers
made exectuion time 5x faster, so that's fairly important to set!
In my case I added/changed some fields to log:
[Date Time] = GETDATE()
[Task ID] = @[System::SourceID]
[Package Duration] / [Container Duration] = (DT_STR, 10, 1252)
(DATEDIFF( "Ms", @[System::StartTime] , GETDATE() )/1000.0)
I.e. make them reals with fractional seconds included
[Read Count] = #imported rows
[Reject Count] = #rejected rows
Any comments or thoughts on all this?
Do you think a custom log provider that picks up row count variables
etc. would be a better choice?
Thanks,
Gary
it differently, and only a few grey hairs later it seems to work
fine:-)
1) Set the SQL insert statement(s) as an expression string in a
variable with global scope, but with EvaluateAsExpression = FALSE.
This is the only place I need to set the SQL statement(s):
OnPostExecute_Sql = "INSERT INTO ..." + @[User::InsertRow] + ...
OnError_Sql = "INSERT INTO ..." + @[System::ErrorCode] + ...
2) Create a variable with global scope to hold a list of the
counting variables that needs resetting to 0 after each event:
Log_Variables_To_Reset = User::Delete_Count,User::Insert_Count...
3) In the parallel containers, create local row count variables
Insert_Count..., i.e. one set of variables for each container.
This keeps the containers separate when they're writing to
variables with otherwise the same name.
4) Create an event handler in one of the parallel containers, i.e.
with scope limited to one of the parallel containers, it has:
a) A local variable in the event handler with EvaluateAsExpression
= TRUE
User::OnPostExecute_Sql_Local
b) "Expand OnPostExecute_Sql" script with a single line (this is the
'trick' that does an extra EVALUATE() on the SQL expression string,
I couldn't find another way of doing it, but this works fine.)
Dts.Variables("User::OnPostExecute_Sql_Local").Expression = _
Dts.Variables("User::OnPostExecute_Sql").Value.ToString()
c) The Execute SQL task has a property expression with
SqlStatementSource = @[User::OnPostExecute_Sql_Local]
d) A script task to reset the appropriate variables:
Dim Vars As String = Dts.Variables("User::Log_Variables_To_Reset") _
.Value.ToString()
Dim VarArr() As String = Split(Vars, ",")
For i As Integer = 0 To VarArr.Length - 1
Dts.Variables(VarArr(i)).Value = 0
Next
5) Now, for any new parallel container, copy&paste the container and
the two event handlers, and create just the local variable
OnPostExecute_Sql_Local with EvaluateAsExpression=TRUE in the event handler,
and you're done - no need to edit any SQL or expressions in the event
handlers, and the containers can run and log in parallel.
6) To change the SQL insert statement across all parallel containers,
add additional variables to log etc, simply edit
OnPostExecute_Sql, OnError_Sql, Log_Variables_To_Reset
in one place in the global scope.
One gotcha: Enabling precompiled scripts in the event handlers
made exectuion time 5x faster, so that's fairly important to set!
In my case I added/changed some fields to log:
[Date Time] = GETDATE()
[Task ID] = @[System::SourceID]
[Package Duration] / [Container Duration] = (DT_STR, 10, 1252)
(DATEDIFF( "Ms", @[System::StartTime] , GETDATE() )/1000.0)
I.e. make them reals with fractional seconds included
[Read Count] = #imported rows
[Reject Count] = #rejected rows
Any comments or thoughts on all this?
Do you think a custom log provider that picks up row count variables
etc. would be a better choice?
Thanks,
Gary