set-StrictMode -version 2

$FileAge = 30 #age of files in minutes
$RenameDelay = 5 #delay value for renaming files
$Source = “\\server1\path1\” #source folder
$Destination =”\\server2\path2\” #destination folder

$Past = (Get-Date).AddMinutes($FileAge * -1)
$Past2 = (Get-Date).AddMinutes($RenameDelay * -1)

#first pass - move files form source to destination
$List = dir $Source -recurse | Where {$_.LastWriteTime -lt $Past -and $_.name -notlike “someString*” -and $_.name -notlike “SomeOtherString*”}
if ($list -ne $null){
write-host $list.count “items found to move”
foreach($file in $list){
#$file.Fullname
If ($file.PSIsContainer -eq $false) {
$MoveDirectory = (join-path -path $Destination -childpath $file.DirectoryName.SubString(14))
new-item -path $MoveDirectory -type directory -ErrorAction:SilentlyContinue #create destination folder if it doesn’t already exist
}
$MovePath = (join-path -path $Destination -childpath $file.FullName.SubString(15))
copy-item -Path $file.FullName -destination $MovePath -force #move file to destination folder
}
}
else {write-host “no matching items found to move”}

#second pass - rename files in destination
$List = dir $destination -recurse | Where {$_.CreationTime -gt $Past2 -and $_.PSIsContainer -eq $false -and $_.name -notlike “SomeString*” -and $_.name -notlike “SomeOtherString*” }
if ($list -ne $null){
write-host $list.count “items found to rename”
foreach($file in $list){
$file = ( rename-item -force –path $file.Fullname –Newname ( $file.basename + (get-date -format ‘_yyyy-MM-dd_hhmm’) + $file.extension))
}
}
else {write-host “no matching items found to rename”}

set-StrictMode -Off

Basically, you use LogParser to create a two-column table containing file paths and MD5 hashes like this:

LogParser “SELECT path, HASHMD5_FILE(Path) AS MD5 INTO T_FILE_INFO FROM D:\some_path\*.*” -i:FS  -o:SQL  -database:files -driver:”SQL Server”  -createTable:ON

The -o parameter directs LogParser to write the output to a SQL Server database table.

The T-SQL  ranking function makes it very easy to find and pick out duplicate rows.  Instead of finding duplicate rows, I use it to generate statements to delete duplicate files.  The entire script looks something like this:


USE [TempDB]
GO

– Create table to store file paths and MD5 hashes
IF  EXISTS (SELECT * FROM sys.objects WHERE OBJECT_ID = OBJECT_ID(N’[dbo].[#T_FILE_INFO]’) AND TYPE IN (N’U’))
DROP TABLE [dbo].[#T_FILE_INFO]
GO
CREATE TABLE [dbo].[#T_FILE_INFO](
[PATH] [varchar](512) NULL,
[MD5] [varchar](255) NULL
) ON [PRIMARY]
GO
– run LogParser to populate table.  Either run from CMD prompt or use XP_CMDSHELL

– note: this step can be time consuming, since LogParser must generate an MD5 has for every file specified

EXEC xp_cmdshell @LogParserCMD;

SELECT ‘DEL “‘ + PATH + ‘”‘
FROM tblCTE
WHERE Ranking > 1
AND MD5 IS NOT NULL;
– Review the result set and run it from a command prompt

– clean up after yourself
DROP TABLE #T_FILE_INFO

In its time it was the worlds largest floating data center with all sorts of cool computers and electronic instrumentation.   It performed bathymetric, magnetic field, and gravity surveys in support of the Navy’s FBM program.  The crew consisted of roughly 40 civilian mariners, 10 oceanographers, two commissioned naval officers, 18 enlisted Navy technicians, and several of defense contractor represenatives.

The ship was big, comfortable and well-appointed.  There were three dining rooms, two libraries, a conference room that doubled as a movie theater, nice staterooms, a workout room, a crew’s lounge and a large basketball court.   My favorite parts of the ship were the electronics shop - I pretty much had it to myself and built my first Z80-based computer there - and the bow, which offered sun, fresh air, and quiet solitude - a great place to study and contemplate life.

The photo below is a webcam capture that shows the Hess entering the Gatun locks of the Panama Canal on 2/18/2011.

The people over at SQLSkills.com have made their MCM training videos available online for free.  They’re pretty incredible.  Subject areas include database internals and performance, high availability and disaster recovery, performance tuning,  security and development support.  Link here:  http://www.sqlskills.com/T_MCMVideos.asp

There are also a plethora of wonderful videos over at SQLBits.com.  One of my favorites is a talk by Thomas Kejser on designing I/O systems for SQL Server.

SQLWorkshops.com has a collection of wonderful videos on advanced query tuning.   The instructor provides live demos with  runtime statistics to fully demonstrate his tuning techniques.  Free and very worthwhile.

The essence of the script follows.  You would might want to modify it to iterate through a list of instances.

IF EXISTS
(SELECT * FROM instance.MASTER.sys.sysprocesses
WHERE cmd = ‘backup database’
AND program_name = ‘SQL Management’)
BEGIN
EXEC msdb.dbo.sp_send_dbmail
@profile_name = ‘Master’,
@recipients = ‘mailbox@domain.com’,
@body = ‘Backup job is still running on instance’,
@subject = ‘Backup job is still running on instace’,
@importance =  ‘high’;
END


The SMO Database class has a Status property which returns at least two values, “Normal” and “Offline”.  You can use it to avoid trying to get info for offline databases.  For example:

if ( $db.status -eq “Normal”) { #exclude offline databases
#do something

}

– generate system catalog changes
– *** note: need to manually edit destination path volume letter in output ***
SELECT ‘ALTER DATABASE ‘ + DB_NAME() + ‘ SET OFFLINE’

SELECT ‘ALTER DATABASE ‘ + DB_NAME() + ‘ MODIFY FILE
(
NAME = ‘ + NAME + ‘,
FILENAME = N”’+ filename + ”’
);’
FROM dbo.sysfiles
– where filename like ‘i:\%’
– order by size desc

SELECT ‘ALTER DATABASE ‘ + DB_NAME() + ‘ SET ONLINE’

– generate robocopy statements to be executed under command prompt
– *** note: need to manually edit destination path volume letter in output ***
SELECT ‘RoboCopy ‘ +
LEFT(filename,(LEN(filename) - CHARINDEX(‘\’, REVERSE(filename)))+1) + ‘ ‘ +
LEFT(filename,(LEN(filename) - CHARINDEX(‘\’, REVERSE(filename)))+1) + ‘ ‘ +
RIGHT(filename,(CHARINDEX(‘\’, REVERSE(filename)))-1)
FROM dbo.sysfiles

– modify filegrowth
SELECT ‘ALTER DATABASE ‘ + DB_NAME() + ‘ MODIFY FILE
(
NAME = ‘ + NAME + ‘,
Filegrowth = 128MB
);’
FROM dbo.sysfiles


SELECT i.name, i.type_desc, i. is_primary_key, i.is_unique, s.name AS [Filegroup]
FROM sys.indexes i
INNER JOIN sys.data_spaces s
ON i.data_space_id = s.data_space_id
WHERE i.name IS NOT NULL
AND i.name NOT IN (‘clust’, ‘clst’, ‘nc1′, ‘nc2′, ‘nc3′, ‘nc’, ‘cl’)
ORDER BY s.data_space_id, i.name

#Powershell Script to delete files older than a certain age

$intFileAge = 90  #age of files in days

$strFilePath = “c:\archive” #path to clean up

#create filter to exclude folders and files newer than specified age

Filter Select-FileAge 

      {

      param($days)

      If ($_.PSisContainer) {}

              # Exclude folders from result set

      ElseIf ($_.LastWriteTime -lt (Get-Date).AddDays($days * -1))

            {$_}

      }

      get-Childitem -recurse $strFilePath | Select-FileAge $intFileAge ‘CreationTime’ | remove-item 

To do: come up with a version that uses LogParser to do the same thing.

DECLARE @SQL VARCHAR(MAX)
CREATE TABLE #temp1 (DRIVE CHAR(1), MB_FREE INT)
INSERT INTO #temp1 (DRIVE, MB_FREE)
EXEC xp_fixeddrives
CREATE TABLE #TEMP2(SUBDIRECTORY VARCHAR(256),
DEPTH INT, [FILE] INT, FULL_PATH VARCHAR(256), COMMAND VARCHAR(512))
SELECT @SQL=COALESCE(@SQL,”)+ ‘INSERT INTO #TEMP2 (SUBDIRECTORY, DEPTH, [FILE]) EXEC Master.dbo.xp_DirTree “‘ + DRIVE +
‘:\MSSQL\DATA”, 1, 1; UPDATE #TEMP2 SET FULL_PATH = ”’ + DRIVE + ”’ + ”:\MSSQL\DATA\” + SUBDIRECTORY WHERE FULL_PATH IS NULL; ‘
FROM #temp1
PRINT @SQL
EXEC (@SQL)
DELETE
FROM #TEMP2
WHERE [FILE] = 0
OR SUBDIRECTORY LIKE ‘%CER’
OR SUBDIRECTORY LIKE ‘MSSQLSYSTEMRESOURCE%’
OR SUBDIRECTORY LIKE ‘distmdl%’
OR FULL_PATH IN
(
SELECT PHYSICAL_NAME
FROM SYS.MASTER_FILES
)
SELECT FULL_PATH, ‘DEL ‘ + FULL_PATH AS COMMAND
FROM #TEMP2
DROP TABLE #temp1
DROP TABLE #TEMP2