Sunday, 10 May 2020

TechNet Gallery to GitHub Migration Tool


As you may or may not be aware Microsoft are making a move to close down the TechNet Gallery web site. This website is the location where I had previously posted all of my tools over the past few years and is also jam packed full of other great community tools as well. In the first stage of the shutdown, Microsoft has put the TechNet Gallery pages into a Read-Only mode where you can still edit existing posts but cannot create new posts. This first step is to give people time to download their posts and move them to another location before the final shutdown of the site.

I find this news to be quite concerning because I know that I still find useful resources on TechNet Gallery all the time. So I've decided to try and do something to help people move their content off TechNet gallery to a safe harbor before it's too late. To do this, I decided to put together a PowerShell tool that will both download content from TechNet Gallery and prepare it for upload directly onto GitHub. This post also describes the steps to uploading the files onto GitHub to make the process as easy as possible.

How does the tool work? Well, you feed it one or many TechNet Gallery URL(s) and it will download the posted file and all the text and images from the site into a folder for you. In addition to this, it will convert all of the text into a Markdown based README.MD file (and refactoring image URLs to move to GitHub) so it can be directly uploaded to GitHub to display the exact text that you had on your TechNet Gallery post (minus any code snippets). All you need to do after this is follow a simple process of creating a new GitHub repository and upload these files to it. You will then end up with your TechNet Gallery content re-hosted in a very similar format on GitHub.


The Differences Between TechNet Gallery and GitHub


TechNet Gallery was designed to be a relatively simple CMS style system that allowed you to upload a file (of any type) and put some blog post style text and images with the file to explain whatever you wanted about the file. This concept is fairly easy to understand and didn’t offer too many bells and whistles. GitHub on the other hand was designed to be a source code repository with versioning features and functionality built in. As a result, GitHub has a much bigger capability set and complexity level than TechNet Gallery.

What does that mean when moving files and content over from TechNet Gallery to GitHub? It means that you have more options for how you can post and manage your files. For example, you could simply use it as a location to put your files or you could potentially start using the Git application workflow to manage the versioning of your files. Using these more complex components of GitHub is out of scope for this blog post. You could quite easily “git init” the folder that the tool creates and then link this up to a remote GitHub repository as a way to upload the content. I will assume if you know how to use Git then you’ll be more than capable of doing this yourself, and if not, then you don't have to use these features or you can read up on them as you see fit.


Backup / Migration Procedure


For the purposes of this blog post I am going to make this the simplest procedure possible which does not require that you install and use Git. Instead the process will use the GitHub web interface to upload your files for storage on GitHub in a similar way to how you would have used TechNet Gallery.

Firstly you need to go and get the URL of your TechNet post. This will look something like this one for one of my posts:

https://gallery.technet.microsoft.com/Teams-Direct-Routing-Tool-42c0bdef

Once you have this you will need to sign up for a GitHub account. This is free and offers you a heap of functionality without asking you any of your hard earned dollars (thank you Microsoft!). I’m not going to show you how you do this because it’s pretty straight forward. When you sign up you will be asked to create a GitHub account name. You will require this account name if you are migrating your images over to GitHub using the tool.


Head over to my GitHub repository and grab of a copy of the script I wrote for downloading the files and generating a README.md file for use on GitHub. Click the link below to go to the repository:




DOWNLOAD SCRIPT HERE


When you run the tool you can either use PowerShell flags to input your information into the tool or not use any flags and the script will ask you to input the information that's required. The flags that the tool uses are as follows:

-MoveImagesToGitHub (Default TRUE) - When set to $false you can retain the original location of the images from the TechNet post in the GitHub readme.md file. If these originally pointed to an external location then you can keep them there but if you uploaded them to TechNet Gallery you will want to move them as they will disappear when the site is taken down.

-TechNetURLs - This flag is where you can input a comma separated list of all the TechNet URLs that you want to download. If you don't use this flag the tool will ask you for a URL when it runs.

-GitHubAccount -  This flag is used for updating image URLs with the GitHub account location where it's going to be stored. This will be the account name that you sign into GitHub with. If you don't use this flag and the script determines that it's required it will ask you for it.

-GitHubRepo (Default TechNet Post Title) - This is the Repo name that the files will be uploaded to. If you don't enter this then the TechNet title will be used. If you have a specific name you want to use then use this flag to set it.

To run the script with flags you will use the following format:
.\TechNet-Gallery-to-GitHub-Migrator.ps1 -TechNetURLs "https://gallery.technet.microsoft.com/Teams-Direct-Routing-Tool-42c0bdef", "https://gallery.technet.microsoft.com/Lync-Skype-for-Businesss-d422212f" -GitHubAccount "<your account name>"

After you have run the script it will generate a new folder for each TechNet Gallery post in the root folder that you ran the script from. These folders will be named using the title of your TechNet post with dashes replacing spaces. This is a common naming convention for GitHub repositories (not the only way, but a common one). The script is designed based on the assumption that the repository that you create in GitHub will have this same name (and same format).



Sign into GitHub with your account. Within GitHub go to your Repositories page. In here you will see a green button marked “New”. Click this button to create a new repository:



To name the repository, copy and paste the name of the folder that the tool created (this is the same as the TechNet Gallery post title). I recommend you use this same name for your GitHub repository when you create it (unless of course you wrote weird or super verbose things in your title to try and up your SEO). Below I have copied the name of the folder:



The New Repository page on GitHub asks for the following information:



Fill in the repository name and description with the folder name that the tool created. Select “Public” repository type if you want others to be able to see it on the Internet. Once you have done this click the “Create repository” button.

You will now be taken to a page that tells you how you can connect to this repository using Git methods. However, we are just going to use an entirely web based creation process for simplicity. Click the “Uploading an existing file” link as highlighted below:



You will now be taken to a page for uploading your files. Drag and drop all the files from the folder (the script created) on the web browser window in the location shown below:



You should now see a list of your files in the page as seen below. Click the “Commit changes” button to upload the files:




You will see your newly created repository with all your files in it. The contents of the README.md file will be displayed in all its glory below this as GitHub uses this as your repository’s information text by default. It should also render all the Markdown in the file nicely with headings, bolding, dot point and images being displayed. This should make the page look similar to what you would have seen on the TechNet Gallery page for your content!





 Tweaking or Changing the README.md Text


Minor tweaking may be required on the Readme.md document to perfect the layout. The good news is you can do any corrections you need to do in the GitHub web editor. Simply click on the README.md file and then click the Edit button. This will take you to a text based editor in the web page where you can make your changes. In this example the Protocol Documentation heading is not quite rendering properly as being bold because there is a return carriage between the second set of asterisks. So I just need to simply move them up to the line that they heading is on and click the Commit Changes button at the bottom of the page.

Note: Code snippets in the TechNet Gallery text will not be carried over to the README.md file (lots of people include their entire scripts in the body text which doesn't translate well to markdown). If you have put smaller lines of code examples in your text you will have to add them manually to the README.md file.




And you’re done!


What Next?


The good news is that you have a publicly available backup of your TechNet Gallery content. It's up to you on how you would like to administer this moving forward on GitHub. As I mentioned earlier, you may choose to start using the Git workflow and downloading and managing your repo(s) with Git commands (commit, push, etc). You may also choose to use the Releases page in your repo to generate tagged and versioned releases. It will depend on the type of project that you are migrating as to how you may choose to do this. For a project that is not getting changed much you may just choose to put up and leave it. However, for a project that you are actively doing a lot of updates on you may choose to start using Git. There are many resources on the web that will explain Git and how it works. Git will be a lot to digest when you first start using it and it's something that you really need to start using in practice to get your head around. As they say; there's no better time to start something than right now, so hop to it!


The Wrap Up


I have migrated 14 of my tools that were up on TechNet Gallery using this process so the process has been trialed and worked successfully for me. As I mentioned earlier in the post; TechNet Gallery has loads of great content on it which I really hope does not disappear into the abyss when the site is closed down because people put the migration in the too hard basket. I really hope that this tool and blog post do something to make the process a bit simpler and give you a framework by which to migrate your tools as well. If you have any issues with the script, let me know.


Read more →

Popular Posts