Improvised support for long paths in Notepad++

I have always been a Windows guy, so this tends to remain my primary desktop environment, even when working on code that ultimate gets run on other platforms.

Even though much of my development happens in Visual Studio Code these days (or an agentic fork of VSCode), I still perform a lot of log file parsing, JSON debugging and ad-hoc regex formatting with the Notepad++ editor for Windows - probably just because I tend to be quite conservative with text editors that I have developed some "muscle memory" for.

One very particular "itch" with Notepad++ comes from my preferred combination of using it as the primary text editor for the WinSCP file manager on a PC. The following issue may seem obscure at first, but it flows naturally from the specific way how WinSCP deals with temporary files:

If you double click on a file to edit, its local download path consists of the Windows %TEMP% prefix plus by a directory name reflecting the process ID of WinSCP, followed by the full path name on the Unix host. This is nice because the temp name reminds you exactly of where your file came from, but it can lead to extremely deep temp structures, if that's what your host looks like. This can be alleviated a little by manually editing the TEMP environment variable to point to something short, such as c:\temp, but even this only goes so far - at some point, you will go over the 260 character MAX_PATH limit of Windows pathnames.

This limit can be stretched by enabling longname support in Windows using this registry setting:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem]
"LongPathsEnabled"=dword:00000001

When this setting is enabled and you double-click a file with a longer path in Explorer, Windows usually tries to shorten pathnames to 8+3 format (AVERYL~1.TXT instead of a very long filename.txt) before passing them to an application that is not marked as supporting long paths in its manifest file, such as Notepad++. This makes files with long paths accessible to an app from Explorer, as long as the converted form does not itself exceed the 260 character limit.

However, this does not help for the WinSCP, which passes the long name directly on the command like when invoking Notepad++ as a viewer for an associated file extension (e.g. `.txt` or `.log`). This leads to Notepad++ opening with an empty file whenever the filename exceeds the limit.

The workaround is to write a one-line batch file (say, nppwrap.cmd) that processes the pathname with a special modifier %~s to return the shortened 8+3 representation, and to invoke Notepad++ with that:

start "" "\Program Files\Notepad++\notepad++.exe" %~s1

You can then associate any extension with this wrapper instead of Notepad++ itself in the WinSCP configuration, and it will let you double-click on the file and open it as usual (except for showing the 8+3 version of the name).

This trick can probably also be used for other viewers that choke on long filenames.

An alternative could be a function such as this (based on the implementation in Notepad++ and extended to support longer path) which makes a file path absolute if needed, while at the same time keeping it within the 260 character limit where possible. It does so by using the "magic" \\?\ prefix that overrides the normal length limit in specific API calls for apps that are not marked as supporting long paths everywhere:

wstring relativeFilePathToFullFilePath(const wchar_t *relativeFilePath)
{
	wstring fullFilePathName;
	BOOL isRelative = ::PathIsRelative(relativeFilePath);

	if (isRelative)
	{		
		// First call to get the required buffer size
		DWORD requiredSize = ::GetFullPathName(relativeFilePath, 0, NULL, NULL);
		if (requiredSize > 0)
		{
			// Second call with dynamically allocated buffer of exact size needed
			std::vector<wchar_t> fullFileName(requiredSize);
			DWORD len = ::GetFullPathName(relativeFilePath, requiredSize, fullFileName.data(), NULL);
			if (len > 0)
			{
				fullFilePathName = fullFileName.data();
			}
		}
	}
	else
	{
		fullFilePathName += relativeFilePath;
	}

	if (fullFilePathName.size() >= MAX_PATH - 1)
	{
		// Attempt to reduce long pathname by getting its short path form
		std::vector<wchar_t> shortPath(MAX_PATH);
		if (fullFilePathName.compare(0, 2, L"\\\\") == 0)
		{
			fullFilePathName.insert(2, L"?\\UNC\\");
		}
		else
		{
			fullFilePathName.insert(0, L"\\\\?\\");
		}
		DWORD len = ::GetShortPathName(fullFilePathName.c_str(), shortPath.data(), MAX_PATH);
		if (len > 0 && len < MAX_PATH)
		{
			fullFilePathName = shortPath.data();
		}
		// else keep fullFilePathName as is, will fail later if path is too long
		if (fullFilePathName.compare(0, 4, L"\\\\?\\") == 0)
		{
			if (fullFilePathName.compare(4, 4, L"UNC\\") == 0)
				fullFilePathName.erase(2, 6);
			else
				fullFilePathName.erase(0, 4);
		}
	}

	return fullFilePathName;
}