Showing posts with label multiple. Show all posts
Showing posts with label multiple. Show all posts

Thursday, March 29, 2012

Create an array in a result field

I am between the "newbie" and "intermediate" stages of writing SQL code and I am wondering if there is a way to capture multiple results into one field so I can basically create a "set" for a unique identifier. Here is few result samples I receive from this code I am using now.

ReqNo ProcID

7102005 1409

7102005 1796

7139003 1411

7139003 6097

7261030 1409

7261030 1796

7268303 3998

7268303 4000

I would like to create a single row for each "ReqNo" and have a field that will an array of the "ProcID" results I receive. In other words, for the first "ReqNo" 7102005, can I create a field that will combine the 1409, 1796 into one field? I am trying to capture an array of integers used for that "ReqNo" so I can use that as a unique identifier in a join for another table.

So, ideally my result would be:

ReqNo ProcSet

7102005 1409, 1796

7139003 1411, 6097

7261030 1409, 1796

7268303 3998, 4000

Is this possible?

declare

@.startdate smalldatetime,

@.enddate smalldatetime ,

@.month int,

@.year int

select

@.startdate = dateadd (dd, -7, getdate())

SELECT

@.month = datepart (month, @.startdate),

@.year = datepart (year, @.startdate)

SELECT

@.startdate = convert (smalldatetime, convert(varchar(2), @.month) + "/1/" + convert (varchar(4), @.year))

select

@.enddate = dateadd (dd, 1 , @.startdate)

select distinct

pp_req_no as ReqNo,

pp_cproc_id_r as ProcID

from

risdb_rch08_stag..performed_procedure

(index pp_serv_time_r_ndx)

where

pp_service_time_r between @.Startdate and @.Enddate

and pp_status_v = 'CP'

and pp_rep_id > 0

order by

pp_req_no, pp_cproc_id_r

You could create a function that would concatenate the fields together and return them as a string. This is not really a recommended practice, but can be occassionally useful for presentation. One trick is to use a local variable to accumulate the results. Of course, your function would be specific to this table.

I am not up on 2005 features, but it seems like there is a new unpivot operator that might work.

This demonstrates how to do the accumulation into a local variable.

use Northwind

go

Declare @.result varchar(1000)

Select @.result = Case When @.result Is Not Null Then @.result + ', ' Else '' End + Convert(varchar(10),OrderID)

From Orders

Where CustomerID = 'ALFKI'

Select @.result|||

Confused?

So are you saying that this syntax @.result + ', ' will build the results in the field? This looks like it will concatenate one result followed by the OrderID in your example (123456, 99999).

How will that build multiple results in one field? I understand how to declare and select, but this does not seem to make sense to me. I will be the first to admit if I am missing something here...which could very well be the case. Bear with me.

|||

Hi,

SELECT t3.ReqNo, MAX(case t3.seq when 1 then t3.ProcID end)

+ MAX(case t3.seq when 2 then ', ' + t3.ProcID else '' end)

+ MAX(case t3.seq when 3 then ', ' + t3.ProcID else '' end) AS ProcID

FROM ( SELECT ReqNo, ProcID, (SELECT COUNT(*) FROM yourTable AS t2 WHERE t2.ReqNo = t1.ReqNo and t2.ProcID <= t1.ProcID) AS seq

FROM yourTable AS t1

) as t3

GROUP BY t3.ReqNo

Or you can use this for SQL Server 2005:

SELECT t3.ReqNo, t3.[1] + coalesce(', ' + t3.[2], '') + coalesce(', ' + t3.[3], '') AS ProcID

FROM (SELECT ProcID, ReqNo, ROW_NUMBER() OVER(PARTITION BY ReqNo ORDER BY ProcID) AS seq FROM yourTable) as t1

PIVOT (MAX(ProcID) for seq in ([1], [2], [3])) AS t3

Or using CTE in SQL Server 2005:

With MyCTE(ReqNo, ProcID, ProcIDs, myNum) AS

(SELECT a.ReqNo, CONVERT(varchar(50), MIN(a.ProcID)) as col1, CONVERT(varchar(50),(a.ProcID)) as ProcIDs, 1 as myNum

FROM yourTable a GROUP BY a.ReqNo, CONVERT(varchar(50),(a.ProcID))

UNION ALL

SELECT b.ReqNo, CONVERT(varchar(50), b.ProcID), CONVERT(varchar(50), (c.ProcIDs + ',' + b.ProcID)), c.myNum+1 as myNum

FROM yourTable b INNER JOIN MyCTE c ON b.ReqNo=c.ReqNo

WHERE b.ProcID>c.ProcID

)

SELECT a.ReqNo, ProcIDs FROM MyCTE a INNER JOIN (SELECT Max(a1.myNum) as myNumMax, a1.ReqNo FROM MyCTE a1

group by a1.ReqNo) b on a.ReqNo=b.ReqNo AND a.myNum= b.myNumMax ORDER BY a.ReqNo

|||

Sorry that I wasn't very clear.

You would build a User Defined Function that would be specific to the table you are working with and would accept a parameter that would identify the records -- like the customerID in the Northwind case. The function would concatenate the values into the local variable and return that variable. You could then call this function as part of your Select clause.

Performance would likely be a challenge.

Here is a NorthWind example

Drop Function OrderList

go

Create Function OrderList( @.cust varchar(10) )

Returns varchar(1000)

As

Begin

Declare @.result varchar(1000)

Select@.result =

Case When @.result Is Not Null

Then @.result + ', '

Else ''

End + Convert(varchar(10), OrderID )

FromOrders

WhereCustomerId = @.cust

return @.result

End

go

SelectCustomerID,

CompanyName,

dbo.OrderList( CustomerID )

FromCustomers

Tuesday, March 20, 2012

Create 1 connection to execute multiple Packages

I want to create 1 connection because I have to execute multiple packages (2005). But when I go into the execute package task editor, the dropdown only ever has "New Connection". Is this a bug?Connections are package local. One package cannot see another package's connection managers.
We, unfortunately, don't have the notion of a file data source which it sounds like you're looking for. Basically, a way to create a connection that you can share across multiple packages.
You'll need to create a new connection per package.
K|||Kirk, not sure if I explained correctly. I'm trying to have 1 master package that executes child packages all on the same "ETL server". So I want 1 connection to be the "ETL Server" such that it can be changed in 1 place (actually, I would just have a runtime setting change it based on a config) when moving from DEV (each developer would have their own instance) to QA to PROD.

The reason I want many packages is so that multiple ETL developers can work on a component and build it without having to wait. Additionally, if everything was in 1 package, it would be huge.

Is the only way to accomplish this without having to change each package's sql server location is to reference it by its file location?|||OK, a few questions to clarify.

What kind of connection do you want the one connection to be SQL, file?
You have one package that you want to call multiple subpackages?
You want the subpackages to use the one connection?
I'm not sure what you mean by "1 connection to be the 'ETL Server' "

"Is the only way to accomplish this without having to change each package's sql server location is to reference it by its file location?"
I don't understand why you need to change the package's sql server location. Which package's SQL Server location? The parent package or the sub-packages?

I think what's confusing me here is that your description has a lot of moving parts. Can you simplify the problem into two or three root issues. I have a simple mind and can't deal with all the complexity. :)

|||Let me offer a scenario and you can tell me how it doesn't meet your needs.

You can create a parent package. It can call all the other sub-packages. When the subpackages execute, the parent packages connection is available to them. However, not at design time. I think this is the issue you're having. The sub-package doesn't have areference to the parent package at design time, so it doesn't have the connections either.

What I'd suggest is that you create a Data source per machine and have all the connections reference it. You'll still need to create a connection manager in each package, but they will all reference the same data source. Moving to different machines is no problem because you can have a "machine resident" data source file that properly points to your "ETL Server".

Is this what you're after?|||see responses below, thanks for taking the time to respond.

Q. What kind of connection do you want the one connection to be SQL, file?
A. The connection of the SSIS server where the subpackages reside.

Q. You have one package that you want to call multiple subpackages?
A. YES

Q. You want the subpackages to use the one connection?
A. YES

Q. I'm not sure what you mean by "1 connection to be the 'ETL Server' "
A. If I store all the subpackages on SQL Server, I'll bound to have different servers for different environments, etc.

Q.I don't understand why you need to change the package's sql server location. Which package's SQL Server location? The parent package or the sub-packages?
A. sub-packages

I'm not after the connections inside the subpackage from the parent packaeg, nor the connections in the parent package from the subpackage.

What I am looking for is to create 1 connection and tie it to where the subpackages are stored since it needs to be dynamic without having to spin through every execute package task in the master package and change the sql server it resides on at runtime.

Alternatively, I could create a new connection for every execute child package in my master package all pointing to the same SQL Server that is storing them, but its extremely redundant and would only be useful if I'm pointing to (LOCAL) and the user and password never changes.

Why can't the EXECUTE PACKAGE task just use one of the connections available in the "connections managers" collection? I thought that was the reason for the "connection manager" abstraction layer, so that a physical connection can be changed in one place, and all logical connections are automatically redirected?

Any chance of implementing this? Alternatively, implementing a FOR EACH connection enumerator (although, this would be hokey)?

Followup question would be this: If I'm executing a package from another package, what is the performance difference (if any) of executing a package in general stored in the file system VS. stored in sql server? I'm assuming that if its not stored in sql server, it needs to be brought in from the file system and compiled on its 1st use?

My goal is to create an ETL where multiple developers can work on different components at the same time. The easiest way to do this would be to create 1 package per table. A master package would call all the child packages, but the master package needs to know which ETL server all the child packages are stored in, which changes from environment to environment.
|||

"Why can't the EXECUTE PACKAGE task just use one of the connections available in the "connections managers" collection?"
It can, If all your execute package tasks are in the same package, yes, they can all use the same connection manager. Is this not working for you?

"what is the performance difference (if any) of executing a package in general stored in the file system VS. stored in sql server"
No performance difference between the two storage methods once loaded. Packages aren't compiled. They're read into the object model.

"The easiest way to do this would be to create 1 package per table. "
Really? Why is this easiest. Why limit it to one package per table. Why not just have developers develop package files and check them into sourcesafe? Then when they're done developing them, deploy to SQL Server? Not sure I follow this logic.

Please keep the threads cogent by only asking only one question per thread.

K

|||Hi,
I'm coming into this late and I think I may be misunderstanding it all but let me have a punt anyway.

You have 1 master package which is intended to call multiple child packages.
You want 1 connection manager in the master package to be used to call all child packages.
You want to cycle through a list of all the child packages and execute them using 1 Execute Package Task
If that is all correct then I think what you want to do is possible using property expressions.

Have I understood correctly?

-Jamie|||"Why can't the EXECUTE PACKAGE task just use one of the connections available in the "connections managers" collection?"
It can, If all your execute package tasks are in the same package, yes, they can all use the same connection manager. Is this not working for you?

Whew, thats all i need, maybe its a bug - using June CTP. I create a new execute package task and then create a new connection for it. Then I create a 2nd execute package task, and click on the connection dropdown, but all that is available is New Connection.

|||

Ah! You're using SQL Connections. Dah!
Yes, this is a bug!
Thanks, great find. Please file a bug.
K

Sunday, March 11, 2012

CR with Left Joint and where condition

I have two tables like this

Table 1

tid
...

Table 2

tid
type
...

the relation is on the column tid of both the tables. We can have multiple records for a tid in Table2 or we may not have any record too.

I am able to get results from Table1 and Table2 by making left outer join in crystal reports (linking). But i have one problem.
I need to get only records in Table2 whose type (field) is "A". For this is achive I am going to select expert in Crystal reports and did type="A" but the problem is if Table2 is not having any records for a tid that tid record is not displaying. How to solve this issue.
I know if we do some thing like this type (+)="A" it will work but How can I do this in Crystal Reports.

Any help is apperciated.

Thanks
KalikiYou can use the add command property in the database expert and write manually your database query

select t.field1, r.field1, t.field2, r.field2
from tab1 t, tab2 r
where tab1.field3 = tab2.field3 (+)
and tab2.field4 (+) = 'A'

This works fine using CR version 10, if you are using an older version (8.5) you can type the 'and tab2.field4 (+) = 'A'' part of the condition to the SQL-query (DATABASE, EDIT SQL-query).

- Jukka|||Thanks for your response.
I am using 8.5 so i am editing the sql via (DATABASE, EDIT SQL-query).

CR 6.0 multiple DSN queries and joins

Hi all, I'm new to this forum so will try to ask only relevant questions on point. What I am looking to do is query multiple DSN sources and join the results in a consolidated report using CR 6.0. Does anyone know if I can query multiple DSN's with CR 6.0 and if so what might the connection string look like?

Thanks in advance.

Cheers

DanSKIf the DSNs point to different database, I think it is not possible. Export the tables in single database and link them

Thursday, March 8, 2012

CPU usage on SQL server

Hi,
I have a quad processor clustered database server that is experiencing
significant performance problems. There are multiple user databases on ther
server, and the issues manifest themselves in different ways. We have had
users unable to save to a database to complete slow down of the whole server
.
The issues seem to be more prevalent in the morning, but we have had them
throughout the day as well. In monitoring the CPU usage we noticed that mos
t
of the activity is going through only 1 of the CPU's. CPU 0 can peak at 100
%
utilisation, and the other 3 have little or no activity at all. There has
been no changes to any settings and the SQL Server install is pretty
standard. Does anyone have any ideas?Meg wrote:
> Hi,
> I have a quad processor clustered database server that is experiencing
> significant performance problems. There are multiple user databases
> on ther server, and the issues manifest themselves in different ways.
> We have had users unable to save to a database to complete slow down
> of the whole server. The issues seem to be more prevalent in the
> morning, but we have had them throughout the day as well. In
> monitoring the CPU usage we noticed that most of the activity is
> going through only 1 of the CPU's. CPU 0 can peak at 100%
> utilisation, and the other 3 have little or no activity at all.
> There has been no changes to any settings and the SQL Server install
> is pretty standard. Does anyone have any ideas?
Have you assigned SQL Server all CPUs? Are you using MAXDOP 1 option in
your queries or have you set the "max degree of parallelism" option on
the server? I would think SQL Server would use all CPUs even if MAXDOP 1
were used when running multiple queries. It's probably some bad query
that is running on a single CPU that is causing the CPU spike. Many
query operations run on a single CPU. And blocking is probably the
problem, not the CPU per se. Although the high CPU is probably causing
extended locking and blocking problems.
You need to profile your database and see how your queries are
performing and figure out which ones are running for long periods and
locking resources required by other SPIDs.
David Gugick
Imceda Software
www.imceda.com

CPU usage on SQL server

Hi,
I have a quad processor clustered database server that is experiencing
significant performance problems. There are multiple user databases on ther
server, and the issues manifest themselves in different ways. We have had
users unable to save to a database to complete slow down of the whole server.
The issues seem to be more prevalent in the morning, but we have had them
throughout the day as well. In monitoring the CPU usage we noticed that most
of the activity is going through only 1 of the CPU's. CPU 0 can peak at 100%
utilisation, and the other 3 have little or no activity at all. There has
been no changes to any settings and the SQL Server install is pretty
standard. Does anyone have any ideas?Meg wrote:
> Hi,
> I have a quad processor clustered database server that is experiencing
> significant performance problems. There are multiple user databases
> on ther server, and the issues manifest themselves in different ways.
> We have had users unable to save to a database to complete slow down
> of the whole server. The issues seem to be more prevalent in the
> morning, but we have had them throughout the day as well. In
> monitoring the CPU usage we noticed that most of the activity is
> going through only 1 of the CPU's. CPU 0 can peak at 100%
> utilisation, and the other 3 have little or no activity at all.
> There has been no changes to any settings and the SQL Server install
> is pretty standard. Does anyone have any ideas?
Have you assigned SQL Server all CPUs? Are you using MAXDOP 1 option in
your queries or have you set the "max degree of parallelism" option on
the server? I would think SQL Server would use all CPUs even if MAXDOP 1
were used when running multiple queries. It's probably some bad query
that is running on a single CPU that is causing the CPU spike. Many
query operations run on a single CPU. And blocking is probably the
problem, not the CPU per se. Although the high CPU is probably causing
extended locking and blocking problems.
You need to profile your database and see how your queries are
performing and figure out which ones are running for long periods and
locking resources required by other SPIDs.
David Gugick
Imceda Software
www.imceda.com

CPU usage on SQL server

Hi,
I have a quad processor clustered database server that is experiencing
significant performance problems. There are multiple user databases on ther
server, and the issues manifest themselves in different ways. We have had
users unable to save to a database to complete slow down of the whole server.
The issues seem to be more prevalent in the morning, but we have had them
throughout the day as well. In monitoring the CPU usage we noticed that most
of the activity is going through only 1 of the CPU's. CPU 0 can peak at 100%
utilisation, and the other 3 have little or no activity at all. There has
been no changes to any settings and the SQL Server install is pretty
standard. Does anyone have any ideas?
Meg wrote:
> Hi,
> I have a quad processor clustered database server that is experiencing
> significant performance problems. There are multiple user databases
> on ther server, and the issues manifest themselves in different ways.
> We have had users unable to save to a database to complete slow down
> of the whole server. The issues seem to be more prevalent in the
> morning, but we have had them throughout the day as well. In
> monitoring the CPU usage we noticed that most of the activity is
> going through only 1 of the CPU's. CPU 0 can peak at 100%
> utilisation, and the other 3 have little or no activity at all.
> There has been no changes to any settings and the SQL Server install
> is pretty standard. Does anyone have any ideas?
Have you assigned SQL Server all CPUs? Are you using MAXDOP 1 option in
your queries or have you set the "max degree of parallelism" option on
the server? I would think SQL Server would use all CPUs even if MAXDOP 1
were used when running multiple queries. It's probably some bad query
that is running on a single CPU that is causing the CPU spike. Many
query operations run on a single CPU. And blocking is probably the
problem, not the CPU per se. Although the high CPU is probably causing
extended locking and blocking problems.
You need to profile your database and see how your queries are
performing and figure out which ones are running for long periods and
locking resources required by other SPIDs.
David Gugick
Imceda Software
www.imceda.com

Wednesday, March 7, 2012

CPU limitations of SQL express

I had a question about the CPU limitations of SQL express. It says limited to 1 CPU but what if I have multiple instaces since the application is a new service and runs in it 'sown memory space how is it limited to 1 CPU? does the service use multiple processors and the queris are binded to one?

Thanks

Hi,

SQL Server will ignorte the other CPUs. if you have another edition on the server it could keep track of the other CPUs as well, although SQL Server Express is installed.

HTH, Jens SUessmeyer.

http://www.sqlserver2005.de
|||What about multiple instances can I bind instance 1 to cpu1 and instance 2 to cpu 2?|||

Yes you can, but a processor licensing, you have to license all CPUs that are activated in the BIOS even if they are not used of the instance. For the Client/server this doens′t apply.

HTH, Jens Suessmeyer.

http://www.sqlserver2005.de

CPU limitations of SQL express

I had a question about the CPU limitations of SQL express. It says limited to 1 CPU but what if I have multiple instaces since the application is a new service and runs in it 'sown memory space how is it limited to 1 CPU? does the service use multiple processors and the queris are binded to one?

Thanks

Hi,

SQL Server will ignorte the other CPUs. if you have another edition on the server it could keep track of the other CPUs as well, although SQL Server Express is installed.

HTH, Jens SUessmeyer.

http://www.sqlserver2005.de
|||What about multiple instances can I bind instance 1 to cpu1 and instance 2 to cpu 2?|||

Yes you can, but a processor licensing, you have to license all CPUs that are activated in the BIOS even if they are not used of the instance. For the Client/server this doens′t apply.

HTH, Jens Suessmeyer.

http://www.sqlserver2005.de

Sunday, February 19, 2012

counting the years in multiple rows question

am making a CV program and i need a way to count the experience the user has:
i have his begin date and end Date as datetime in an sql server.
i can do it programicly but i prefer to do it at the sql side
the question:
how can i get how much exp he has aka :

Code Snippet

SUM(DATEDIFF(year , JobApExp.BeginDate , JobApExp.EndDate ))

but for all the datarow
(he has more than one BeginDate and EndDate (for each job he has one))

P.S i want to be able to use it in a where clause :

Code Snippet

select * from jobap
where -- or HAVING
JobAp.ind = JobApExp.JobAp AND
SUM(DATEDIFF(year , JobApExp.BeginDate , JobApExp.EndDate )) > CONVERT(int,@.Exp)

thanks in advance

You have to use the grouping together with the having clause:

declare @.Exp as int

select max(A) as A, max(B) as B from jobap

where JobAp.ind = JobApExp.JobAp

HAVING SUM(DATEDIFF(year , JobApExp.BeginDate , JobApExp.EndDate )) > @.Exp

|||


create table #Personnel(id int,name varchar(20))
insert #Personnel select 1,'madhu'
insert #Personnel select 2 ,'zyx'

create table #PersonnelExp (ID int,fmdt datetime,todt datetime)
delete #PersonnelExp
insert #PersonnelExp select 1,'1990-1-1','1999-1-1'
insert #PersonnelExp select 1,'1999-1-2','2002-1-2'
insert #PersonnelExp select 1,'2003-1-2','2007-1-2'
insert #PersonnelExp select 2,'1995-1-1','1999-1-1'
insert #PersonnelExp select 2,'1999-1-2','2004-1-2'
insert #PersonnelExp select 2,'2005-1-2','2007-1-2'

select a.ID,a.Name,b.TotExp From #Personnel a,
(select id,SUM(DATEDIFF(year , fmdt, todt )) As TotExp from #PersonnelExp group by id) b
where a.ID=b.ID
and totexp>12

Check this script

Madhu

|||

You may want to consider performing the DATEDIFF on the day or month level, as the year shows the year difference between the two dates.

IE

select datediff(yy, '01/01/2006', '12/31/2006')

select datediff(yy, '12/31/2006', '01/01/2007')

The first returns 0, even though it's a full year. And the second returns 1 even though it's only been 1 day.

Friday, February 17, 2012

counting multiple values from the same column and grouping by a another column

This is a report I'm trying to build in SQL Reporting Services. I can do it in a hacky way adding two data sets and showing two tables, but I'm sure there is a better way.

TheTable
Order# Customer Status

STATUS has valid values of PROCESSED and INPROGRESS

The query I'm trying to build is Count of Processed and INProgress orders for a given Customer.

I can get them one at a time with something like this in two different datasets and showing two tables, but how do I achieve the same in one query?

Select Customer, Count (*) As Status1
FROM TheTable
Where (Status = N'Shipped')
Group By Customeryou can write a stored proc and use output parameters to return the values..and in your stored proc have multiple sql stmts to get the diff counts..

hth|||This might work for you, there is probably a better way, this can become very expensive with the 2 sub queries if there is a lot of data

 SELECT Customer,
(
SELECT COUNT(*)
FROM TheTable
WHERE Status = N'Shipped'
) AS Shipped,
(
SELECT COUNT(*)
FROM TheTable
WHERE Status = N'SomeOtherStatus'
) AS SomeOtherStatus
FROM TheTable
ORDER BY Customer