Using Visual C# 2010 Express to Check Multiple Folders for Duplicate Files Update

Using Visual C# 2010 Express to Check Multiple Folders for Duplicate Files Update

Since my previous post I have discovered an error such that not all the files of folders get compared.  So an update of the Form1.cs code is provided below.

Also, I have noticed a problem with my attempt to provide the user with a running status of the progress of the compares.  When I ran the application with the debugger the label control that displayed text as to the number of folders checked and the number of files checked and the label control that displayed the number of duplicates continuously updated as intended.  However, when the application was run via its .exe executable the labels would initially be updated.  But very shortly the wait cursor would be displayed without being requested and the updating would cease until all the compares had occurred when the cursor would return to the default and the totals would be displayed.

I have been unable to find a way to avoid this.  I've done internet searches and found various posts that purport to show how the programmer can prevent this from happening – that is, have updates of controls actually happen and be displayed.  None of those that I found have worked with my application.

I've also tried a progress bar which should surely work.  Again, it is continually updated if I run the application via the Visual C# 2010 Express debugger.  But, if the application is run via its executable, the wait cursor appears and the labels and progress bar cease being updated after the first 20 files or so.  So the progress bar no longer shows progress.

In addition I created a second project in which I used a worker thread to do the compares while the Form1 thread slept for 500 ms at a time and then did the updates in the Form1 thread before sleeping again.  No change in the update behavior.  That is, the label text didn't change and the progress bar didn't show any progress.

Any ideas anyone?

Form1.cs code

 using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading;
using System.Windows.Forms;

namespace Compare
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            // (Mostly) generated control initialization.
            InitializeComponent();

            // Add selections to the combo boxes.
            extensionComboBox.MaxDropDownItems = 4;
            extensionComboBox.Items.Add(extensions[0]);
            extensionComboBox.Items.Add(extensions[1]);
            extensionComboBox.Items.Add(extensions[2]);
            extensionComboBox.Items.Add(extensions[3]);

            existingReport.MaxDropDownItems = 2;
            existingReport.Items.Add(reportOptions[0]);
            existingReport.Items.Add(reportOptions[1]);

            // Create string from byte arrays to get new lines in the duplicates file.
            Byte[] cR = new Byte[] { 13 };
            Byte[] lF = new Byte[] { 10 };
            crNL = Encoding.ASCII.GetString(cR) + Encoding.ASCII.GetString(lF);
        }

        //----------------------------------------------------------------------
        // Compare files in the selected folders.
        private void CompareFiles()
        {

            // Display progress bar.
            progressBar.Minimum = 1;
            progressBar.Maximum = 0;
            for (int f = 0; f < folderCount; f++)
            {
                progressBar.Maximum = progressBar.Maximum + 
                                      folderFile[f].list.Count();
            }
            progressBar.Value = 1;
            progressBar.Step = 1;
            progressBar.Visible = true;
            progressBar.Update();

            // Display Please Wait.
            pleaseWait.Visible = true;
            pleaseWait.Refresh();

            // Compare files of all folders starting with first file of first folder
            // to compare with the other files of that folder and then those of the
            // other folders.  Then proceed to the next file of the first folder and
            // repeat.  Then to the first file of the next folder to check the files
            // of that folder and the remaining folders.  And repeat.
            for (int f = 0; f < folderCount; f++) // Check folders for those not yet
            {                                                       //  completely compared
                // Select initial default folder and list.
                int defaultFolder = f;
                IEnumerable<System.IO.FileInfo> defaultList =
                                                folderFile[defaultFolder].list;

                FileInfo lastIndex = defaultList.Last();

                // Ignore list if no files with selected extension.
                if (defaultList.Count() > 0)
                {
                    string defaultFileName = defaultList.First().FullName;
                    FileStream defaultFileStream = null;

                    defaultIndex = 0;
                    defaultCount = 0;
                    if (defaultFiles == null)
                    {
                        defaultFiles = new string[defaultList.Count()];
                    }
                    else if (defaultFiles.Count() < defaultList.Count())
                    {
                        defaultFiles = new string[defaultList.Count()];
                    } // else reuse the defaultFiles array
                    foreach (var v in defaultList)
                    {
                        defaultFiles[defaultCount] = v.FullName;
                        defaultCount++;
                    }

                    // Compare particular file in the list to the other files of the list
                    // and the other lists (if not already in the duplicate table).
                    for (defaultIndex = 0; defaultIndex < defaultCount; defaultIndex++)
                    {
                       // Select the file.
                       defaultFileName = defaultFiles[defaultIndex];

                        // Ignore file if already found as a duplicate.
                        if (!InDuplicateTable(defaultFileName))
                        { 

                            // Open the selected file for Read-Only.
                            try
                            {
                                defaultFileStream = new FileStream(defaultFileName,
                                                                   FileMode.Open,
                                                                   FileAccess.Read);
                                if (defaultFileStream != null)
                                {   // Find any duplicates of the selected file.
                                    CompareSelectedFile(defaultFolder, defaultFileName,
                                                        defaultFileStream );
                                    // Close the selected file.
                                    defaultFileStream.Close();

                                    // Attempt to continue update without showing the
                                    // wait cursor when running exe.
                                    comparedCount++;
                                    CompareStatus.Text = "Folder " + (f + 1) + "  " +
                                                         comparedCount + " files compared";
                                    CompareStatus.Visible = true;
                                    CompareStatus.Refresh();

                                    numberDuplicates.Text = "Number of duplicate files is "
                                                            + duplicatesReported;
                                    numberDuplicates.Visible = true;
                                    numberDuplicates.Refresh();

                                    progressBar.Value = comparedCount;
                                    progressBar.PerformStep();
                                    progressBar.Update();

                                } // end if
                            } // end try
                            catch (Exception ex)
                            {
                                MessageBox.Show("Error: Unable to open first file " +
                                                defaultFileName +
                                                " " + ex.Message);
                            }
                        } // end if (!InDuplicateTable(defaultFileName))
                      
                    } // end loop for each file in current list

                } // end if (defaultList.Count() > 0)

                // Indicate that the folder has been searched.
                folderFile[f].searched = true;

            } // end loop thru selected folders

            // Hide Please Wait and progress bar.
            pleaseWait.Visible = false;
            progressBar.Visible = false;

            // Output count of duplicate files.
            Byte[] text = new UTF8Encoding(true).GetBytes("Number of duplicate files is "
                + duplicatesReported + crNL);
            duplicatesReport.Write(text, 0, text.Length);
            duplicatesReport.Flush();

            // Display Done OK button to allow user to see number of duplicate files.
            // When clicked, the application will be terminated.
            doneButton.Visible = true;

        } // end method CompareFiles

        //----------------------------------------------------------------------
        // Compare particular file in the list to the other files of the
        // selected folder list and the other lists that follow it.
        // Notes: All files in a list previous to the selected folder have
        //        already been compared.
        private void CompareSelectedFile(int selectedFolder, string selectedFileName,
                                         FileStream fs1)
        {
            for (int f = selectedFolder; f < folderCount; f++)
            {   // Check folders for those not yet completely compared
                if (f == selectedFolder)
                {   // Compare from next file forward thru the selected folder
                    for (int index = defaultIndex+1; index < defaultCount; index++)
                    {
                        FileCompare(fs1, defaultFiles[index]);
                    }
                }
                else
                {   // Compare all the files of the folder
                    foreach (var v in folderFile[f].list)
                    {
                       FileCompare(fs1, v.FullName);
                    }
                }

            }  // end loop over folders

         } // end method CompareSelectedFile

        //----------------------------------------------------------------------
        // This method accepts two strings the represent two files to
        // compare. A return value of 0 indicates that the contents of the files
        // are the same. A return value of any other value indicates that the
        // files are not the same.
        // Notes: This method is somewhat modified from a Microsoft example.
        private void FileCompare(FileStream fs1, string file2)
        {
            int file1byte;
            int file2byte;

            FileStream fs2 = null;

            // Open the second file.
            try
            {
                fs2 = new FileStream(file2, FileMode.Open, FileAccess.Read);
            }
            catch (Exception ex)
            {
                if (ex.Message.StartsWith("Access to the path") &&
                     fileError.StartsWith("Access to the path"))
                { // bypass the reporting of the exception
                }
                else
                { // display the first instance of the exception
                    fileError = ex.Message;
                    MessageBox.Show("Error: Unable to open second file " + file2 + " "
                        + ex.Message + " Further such errors will be ignored.");
                }
            }

            // If can't open the file, return since can't compare the files.
            if (fs2 == null)
            {
                MessageBox.Show("ERROR: Couldn't open " + file2);
                return;
            }

            // Check the file sizes. If they are not the same, the files
            // are not the same.
            if (fs1.Length != fs2.Length)
            {
                // Close the file and return since no need to compare the files
                fs2.Close();
                return;
            }

            // Read and compare a byte from each file until either a
            // non-matching set of bytes is found or until all the bytes
            // have been compared.
            int bytesRead = 0;
            do
            {
                // Read one byte from each file.
                file1byte = fs1.ReadByte();
                file2byte = fs2.ReadByte();
                bytesRead++;
            }
            while ((file1byte == file2byte) && (bytesRead <= fs1.Length)) ;

            // Close the file.
            fs2.Close();

            // If the last bytes read are equal, the files are the same.  Add the
            // duplicate file pair to the duplicate array and duplicates report.
            if ((file1byte - file2byte) == 0 )
            {   // add file pair to duplicate array
                if (duplicateCount < maxDuplicates)
                {
                    duplicateFiles[duplicateCount] =
                        new DuplicateFile(fs1.Name, file2);
                    duplicateCount++;
                }
                else if (!maxDuplicatesDisplayed)
                {
                    maxDuplicatesDisplayed = true;
                    MessageBox.Show("ERROR: Maximum number of duplicates of "
                                     + maxDuplicates + " exceeded");
                }

                // Add file pair to duplicates report
                duplicatesReported++;
                Byte[] text = new UTF8Encoding(true).GetBytes(fs1.Name
                                   + " " + file2 + crNL);
                duplicatesReport.Write(text, 0, text.Length);
                duplicatesReport.Flush();
            } // end if

        } // end function FileCompare

        //----------------------------------------------------------------------
        // Return true if file is in the duplicate table. 
        // Note: The name1 column is for the default file for which duplicates
        //       have been found.  The file passed is for a newly chosen file
        //       so could only be a file in the name2 column.
        private bool InDuplicateTable(string file)
        {
            for (int i = 0; i < duplicateCount; i++)
            {
                if (file.Equals(duplicateFiles[i].name2))
                {
                    return true;
                }
            }

            return false;

        } // end function InDuplicateTable

        //----------------------------------------------------------------------

        // Obtain folder names and display in list box and open report file.
        private void ObtainFolderNames(string ext) // ext is selected file extension
        {
            string folder = "";
            bool selectFolder = true;
            System.IO.DirectoryInfo[] dir = new DirectoryInfo[maxFolders];

            // Do not allow the user to create new files via the FolderBrowserDialog.
            folderBrowserDialog1.ShowNewFolderButton = false;

            // Obtain folders to be searched for duplicate files with the
            // selected extension.
            while (selectFolder && folderCount < dir.Length)
            {
                if (folderBrowserDialog1.ShowDialog() == DialogResult.OK)
                {
                    folder = folderBrowserDialog1.SelectedPath;

                    // Check that folder not already selected.
                    bool newFolder = true;
                    for (int f = 0; f < folderCount; f++)
                    {
                        if (folder.Equals(folderFile[f].name))
                        {
                            newFolder = false;
                            break;
                        }
                    }

                    if (newFolder)
                    {
                        folderListBox.Items.Add(folder);
                        folderListBox.Refresh();
                        dir[folderCount] = new System.IO.DirectoryInfo(folder);

                        IEnumerable<System.IO.FileInfo> folderList =
                            dir[folderCount].GetFiles(
                               ext, System.IO.SearchOption.AllDirectories);
                        folderFile[folderCount] = new FolderFile(folder, folderList);
                        folderCount++;
                    } // end if (newFolder)
                }
                else
                {
                    selectFolder = false;
                }
            } // end select folders loop

            folderListBox.Refresh();

            // Quit if nothing selected.
            if (folderCount == 0)
            {
                Console.WriteLine("ERROR: no folders selected");
                this.Close(); // close form
                return;
            }

            // Open disk file to report duplicates.  Check if already available
            // and, if so, ask if to overwrite or add new entries at the end. 
            // If not available, create the file.
            duplicatesReportFileName = folderFile[0].name + "\\DuplicateFiles.txt";
            try
            {
                duplicatesReport = File.Open(duplicatesReportFileName,
                                             FileMode.Open, FileAccess.Read);
                long size = duplicatesReport.Length;
                if (size > 0)
                {
                    duplicatesReport.Close();
                    existingReport.Visible = true;
                    existingReport.Refresh();
                    return; // allow append or overwrite to be selected
                }
                else
                {
                    duplicatesReport.Close();
                    duplicatesReport = File.Open(duplicatesReportFileName,
                                                 FileMode.Append, FileAccess.Write);
                }
            }
            catch //(Exception ex)
            {
                duplicatesReport = File.Open(duplicatesReportFileName,
                                             FileMode.CreateNew, FileAccess.Write);
            }

            // Compare the selected files.
            CompareFiles();

        } // end method ObtainFolderNames

        //----------------------------------------------------------------------
        // Event handlers

        // Hide OK button and then select the folders of the files to be
        // compared and do the compare.
        private void buttonOK_Click(object sender, EventArgs e)
        {
            buttonOK.Visible = false;
            ObtainFolderNames(extension);

        } // end event handler buttonOK_Click

        //----------------------------------------------------------------------
        // Allow operator to terminate
        private void doneButton_Click(object sender, EventArgs e)
        {
            doneButton.Visible = false;
            this.Close(); // close form
            return; // finished running

        } // end event handler doneButton_Click

        //----------------------------------------------------------------------
        // Input file extension to be used and be sure of leading "*." and
        // wait until validated.
        public void extensionTextBox_KeyDown(object sender, KeyEventArgs e)
        {
            extension = extensionTextBox.Text; // input latest text
            if (e.KeyCode == Keys.Enter)       // all chars entered
            {   // edit entered extension
                char[] chars  = new char[extension.Length];
                chars = extension.ToCharArray();
                if (chars[0].Equals('*') && chars[1].Equals('.'))
                { // both leading characters in string
                }
                else if (!chars[0].Equals('.'))
                { // no leading '.' so add it
                    extension = "." + extension;
                    if (!chars[0].Equals('*'))
                    { //--->>> this not going to work???
                        extension = "*" + extension;
                    }
                }
                else
                { // leading '.' so add leading '*'
                    extension = "*" + extension;
                }

                extensionTextBox.Text = extension; // display what will be used
                buttonOK.Visible = true; // to allow validation of the extension wanted
            } // end if (e.KeyCode == Keys.Enter)

        } // end event handler extensionTextBox_KeyDown

        //----------------------------------------------------------------------

        private void folderListBox_SelectedIndexChanged(object sender, EventArgs e)
        {

        } // end event handler folderListBox_SelectedIndexChanged

        private void CompareStatus_Click(object sender, EventArgs e)
        {

        } // end event handler CompareStatus_Click

        //----------------------------------------------------------------------
        // Obtain the selected extension and display and wait until validated.
        private void extensionComboBox_SelectedIndexChanged(object sender, EventArgs e)
        {
            int index = extensionComboBox.SelectedIndex;
            extension = extensions[index];
            extensionTextBox.Text = extension; // display what will be used
            buttonOK.Visible = true; // to allow validation of the extension wanted

        } // end event handler extensionComboBox_SelectedIndexChanged

        //----------------------------------------------------------------------
        // Obtain the selected option and open append or open create as requested.
        private void existingReport_SelectedIndexChanged(object sender, EventArgs e)
        {
            int index = existingReport.SelectedIndex;
            if (index == 0) // Append
            {
                duplicatesReport = File.Open(duplicatesReportFileName, FileMode.Append,
                                             FileAccess.Write);
            }
            else
            {
                duplicatesReport = File.Open(duplicatesReportFileName, FileMode.Create,
                                             FileAccess.Write);
            }
            existingReport.Visible = false;

            // Compare the selected files.
            CompareFiles();

        } // end event handler existingReport_SelectedIndexChanged

        //----------------------------------------------------------------------
        // Class variables

        // File extension to use for files to be compared
        string extension = "";
        string[] extensions = new string[] { "*.jpg", "*.bmp", "*.doc", "*.txt" };

        string[] reportOptions = new string[] { "Append", "Overwrite" };

        const int maxFolders = 15;
        const int maxDuplicates = 300;
        string crNL = null;

        string fileError = "";

        public struct FolderFile
        {
            public IEnumerable<System.IO.FileInfo> list; // list of file names
            public string name;            // folder name
            public bool searched;          // whether folder has been searched
   
            public FolderFile(string name, IEnumerable<System.IO.FileInfo> list)
            {   // add new folder name and its file list to the structure
                this.name = name;
                this.list = list;
                this.searched = false;
            } // end constructor
        }

        // Files of the currently selected folder
        private int defaultIndex = -1;
        private int defaultCount = 0;
        private string[] defaultFiles = null; //string[];

        public struct DuplicateFile
        {
            private string name1;
            public string name2;

            public DuplicateFile(string name1, string name2)
            {   // add new pair of duplicated files to the structure
                this.name1 = name1;
                this.name2 = name2;
            }
        }

        // Number of selected folders and array of selected folders
        private int folderCount = 0;
        private FolderFile[] folderFile = new FolderFile[maxFolders];

        // Number of compared file pairs
        private int comparedCount = 0;

        // Number of duplicate file pairs and array containing them
        private bool maxDuplicatesDisplayed = false;
        public int duplicateCount = 0;
        public DuplicateFile[] duplicateFiles = new DuplicateFile[maxDuplicates];
       
        // Disk file text writer of the duplicate file pairs to examine when finished
        private int duplicatesReported = 0;
        private string duplicatesReportFileName = "";
        static private FileStream duplicatesReport = null;

    } // end class Form1

} // end namespace Compare