SSIS -How to Get Most Recent File from Directory and Load to a Table [SSIS Script Task]

Scenario:
We get the files in our Source Folder all day long. Each file is appended copy of previous file. We
want to create a SSIS Package that will load only the most recent file to our table.

Solution:
We will be using Script task to find out the name of the most recent file and Data Flow Task to load
that file to our table.Test_File_20170728.txt is the file that should be read as it is most recent file.

Step 1:
Create a variable VarFolderPath that will contain the folder path in which our files exist and
second variable with name VarFileName which will hold the value of most recent File Name.

Step 2:
Drag Script Task to Control Flow Pane and provide the variables to it as shown

Click on Edit Script and write below script. I have only added the code which is in Red.
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.IO;

namespace ST_2650e9fc7f2347b2826459c2dce1b5be.csproj
{
[System.AddIn.AddIn("ScriptMain", Version = "1.0", Publisher = "", Description = "")]
public partial class ScriptMain :
Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
#region VSTA generated code
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
#endregion

public void Main()

{
// TODO: Add your code here
var directory= new DirectoryInfo(Dts.Variables["User::VarFolderPath"].Value.ToString());

FileInfo[] files = directory.GetFiles();

DateTime lastModified = DateTime.MinValue;
foreach (FileInfo file in files)
{
if (file.LastWriteTime > lastModified)
{
lastModified = file.LastWriteTime;
Dts.Variables["User::VarFileName"].Value = file.ToString();
}
}
Dts.TaskResult = (int)ScriptResults.Success;
}
}
}

Step 3:
Let's run our SSIS Package to make sure that our script is returning us expected file name( Most
recent modified by)
Step 4:
Next step will be to load this file to our destination table. Bring Data Flow Task to Control Flow
Pane and connect Script Task to it. Drag Flat File Source inside Data Flow Task and make
connection to any file in the Source Folder as all the files have same structure.

Step 5:
As latest file name will be changing, lets configure our Flat File Connection Manager to use
Filename variable.

Final Output:
Flat File source connect to OLEDB Destination in that Select Database and Table If exist otherwise
Create new table with name. Then mapping the columns.
Moving File:
Moving that File into other folder

SSIS - How to Delete Top N Rows from CSV or Text File in SSIS by using Script Task
Scenario:
We have received text or csv file. The file has some company information, date created etc.
before the data rows start. Our goal is to delete these information rows and regenerate file start
from header row.

Fig: Source File With Company Information

Solution:
We will be using Script Task to Delete the Top N rows from text file. We will be reading all the
data from the file in string type array and then overwrite the original file after removing Top N
rows.
Step 1:
Create two variables as shown in figure.
VarFilePath is variable holding the file path. VarDeleteTopNRows is variable that will hold the number
of rows you want to delete from starting of file.

Fig 1: Create variables in SSIS Package To Delete Top N Rows from Flat File by using Script Task
Step 2:

Bring the Script Task to Control Flow Pane. Map the above Variables as shown to ReadOnlyVariables
in Script Task.

Fig 2: Map Variables in Script Task

Paste the below Code. I have bold the code that I wrote, rest of the code is auto generated by Script
Task.
#region Namespaces

using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.IO;
using System.Linq;

#endregion
namespace ST_c62f3dcfb0964917aade179aac4edfab
{
[Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
public partial class ScriptMain :
Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
public void Main()
{
// TODO: Add your code here
string FilePath = Dts.Variables["User::VarFilePath"].Value.ToString();
Int32 DeleteTopNRows = Convert.ToInt32(Dts.Variables["VarDeleteTopNRows"].Value);
string[] lines = System.IO.File.ReadAllLines(FilePath);
lines = lines.Skip(DeleteTopNRows).ToArray();
System.IO.StreamWriter file = new System.IO.StreamWriter(FilePath);
foreach (string line in lines)
{
 // MessageBox.Show(line.ToString());
file.WriteLine(line);
}

file.Close();
Dts.TaskResult = (int)ScriptResults.Success;
}
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
}
}
Save the code. Run the script task and once completed. Check the file again.

Fig 3: Top 3 Rows Deleted By SSIS Package.

You can change the value of variable VarDeleteTopNRows according to your requirements.

SSIS- How to restart package from failure

SSIS provides a Checkpoint capability which allows a package to restart at the point of failure. The
Checkpoint implementation writes pertinent information to an XML file while the package is executing
to record tasks that are completed successfully and the values of package variables so that the
package's "state" can be restored to what it was when the package failed. When the package completes
successfully, the Checkpoint file is removed; the next time the package runs it starts executing from the
beginning since there will be no Checkpoint file found. When a package fails, the Checkpoint file
remains on disk and can be used the next time the package is executed to restore the values of package
variables and restart at the point of failure.

1. CheckpointFileName:
2. CheckpointUsage:
Never:
IfExists:
Always:
3. SaveCheckpoints:
4. FailPackageOnFailure - Choose from these options: True or False (default). True indicates that
the SSIS package fails if this task fails; this implements the restart at the point of failure
behaviour when the SSIS package property SaveCheckpoints is true and CheckpointFileUsage is
IfExists.
5. FailParentOnFailure - Choose from these options: True or False (default). Select True when the
task is inside of a container task such as the Sequence container; set FailPackageOnFailure for
the task to False; set FailPackageOnFailure for the container to True.