AppendTargetFrameworkToOutputPath not respected by PackageFile target

[jzabroski]

Describe the Bug

AppendTargetFrameworkToOutputPath not respected by PackageFile target, such that the output from dotnet nugetize custom.csproj is as follows:

Package: Custom.0.0.0.1-dev.nupkg
         D:\Users\John.Zabroski\source\infrastructure\trunk\Prototype\bin\Custom.nuspec
    Authors    : John.Zabroski
    Copyright  : Simmaculate c 2021
    Description: Package Description
    Icon       : icon.png
    Version    : 0.0.0.1-dev
  Contents:
    /contentFiles/
      any/
        net5.0/
          ThirdParty.OData

Steps to Reproduce

I don't know how to provide a unit test for MSBuild related stuff.

<?xml version="1.0" encoding="utf-8"?>
<Project Sdk="Microsoft.NET.Sdk">
  <!-- ReSharper disable UnknownProperty -->  
  <PropertyGroup>
    <Authors>John Zabroski</Authors>
    <Copyright>Simmaculate</Copyright>
    <TargetFramework>net5.0</TargetFramework> <!--netcoreapp2.1, netcoreapp3.1 also supported -->
    <EnablePackInference>false</EnablePackInference>
    <PackageIcon>icon.png</PackageIcon>
    <PackageId>Custom</PackageId>
    <AssemblyTitle>Custom Packaging</AssemblyTitle>
    <AppendTargetFrameworkToOutputPath>false</AppendTargetFrameworkToOutputPath>
    <OutputPath>bin</OutputPath>
  </PropertyGroup>

  <PropertyGroup>
    <PackDependencies>true</PackDependencies>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Nugetizer" Version="*" />
    <PackageReference Include="ThirdParty.OData" Version="*" >
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
  </ItemGroup>

  <ItemGroup>
    <PackageFile Include="ThirdParty.OData" Version="*" PackFolder="Content" Pack="true" />
  </ItemGroup>

</Project>

Expected Behavior

Based on AppendTargetFrameworkToOutputPath=false and PackDependencies=true and OutputPath=bin, I would expect the third party nuget package to be transcluded into the bin folder, and the /any/net5.0 sub-tree to be deforested so that its just "/contentFiles/".

Exception with Stack Trace

n/a

Version Info

dotnet-nugetize 0.7, Nugetizer 0.7

Additional Info

I figured I would use this prototype as research into @kzu suggesting I give Nugetizer a try to solve problems with SDK Pack, but so far I am not fully wrapping my head around how this is supposed to work, and I am most definitely not an MSBuild or Nuget expert.

[jzabroski]

Changing

  <ItemGroup>
    <PackageFile Include="ThirdParty.OData" Version="*" PackFolder="Content" Pack="true" />
  </ItemGroup>

to:

  <ItemGroup>
    <PackageFile Include="ThirdParty.OData" Version="*" PackFolder="Content" Pack="true" FrameworkSpecific="false" />
  </ItemGroup>

Generates the following instead:

Package: Custom.0.0.0.1-dev.nupkg
         D:\Users\John.Zabroski\source\infrastructure\trunk\Prototype\bin\Custom.nuspec
    Authors    : John.Zabroski
    Copyright  : Simmaculate c 2021
    Description: Package Description
    Icon       : icon.png
    Version    : 0.0.0.1-dev
  Contents:
    /contentFiles/
      any/
        any/
          ThirdParty.OData

Not exactly what I was expecting.

[kzu]

If you want to precisely control target location, use PackagePath metadata instead. PackFolder applies default heuristics according to the folder default behavior, like what you're seeing. Content/ContentFiles in particular, has specific rules outlined in the nuget docs.

[jzabroski]

I did find PackagePath but I am not sure I understand how to properly use it. Can you point me to the nuget docs that have the specific rules for Content/ContentFiles? In the SDK Pack world, all I have to know about is whether an item is Content or not, so I am not familiar with specific rules, as I am guessing SDK Pack hides them from me?

I think my confusion comes from trying to follow this example on your blog post:

    <!-- Add package dependencies directly too -->
    <PackageFile Include="Newtonsoft.Json" Version="13.0.1" PackFolder="Dependency" />

I didn't realize that what you were implying is I could avoid nuget restore entirely by simply forcing the dependency information. I thought PackageFile Include="Newtonsoft.Json" might somehow automagically download the package and provide us with the option to include all packagereferences in a single package, similar to what Cake and OctoPack do.

[kzu]

https://devblogs.microsoft.com/nuget/nuget-contentfiles-demystified/
https://docs.microsoft.com/en-us/nuget/reference/nuspec#including-content-files

[jzabroski]

OK, I read that, but the behaviors I am seeing still seem broken to me. Here is an example.

I have a hand-written package.nuspec file that looks like this:

<?xml version="1.0" encoding="utf-8" ?>
<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
  <metadata>
    <id>$Id$</id>
    <version>$Version$</version>
    <description>$Description$</description>
    <authors>$Authors$</authors>
    <copyright>$Copyright$</copyright>
  </metadata>
  <files>
    <file src="bin\release\**\*" target="" />
    <file src="*.config" />
    <file src="*.xml" />
  </files>
</package>

When I write the following Nugetizer MSBuild XML, why does it create any/any subdirectories, even when I explicitly forget the target to be ""?

What is the Nugetizer PackageFile equivalent for my <file src="bin\release\**\*" target="" /> line? How do I port my working nuspec file to Nugetizer?

Note, I have:

1. Set <AppendTargetFrameworkToOutputPath>false</AppendTargetFrameworkToOutputPath> to false
2. Set <PackageFile FrameworkSpecific="false" ... />

It seems like there is no way to disable the generation of contentFiles/any/any (although, I do recall yesterday somehow doing it, I cant seem to replicate that success today, and I didnt commit my work yesterday and blew it all away to start over).

[jzabroski]

@kzu thanks for converting this to a discussion.

W.r.t. to mapping my old nuspec file and <file src="bin\release\**\*" target="" /> to PackageFile element, how do I do that?

[jzabroski]

It also looks like ContentTargetFolders isn't respected. Not sure if it should be. Mostly just going off the description that Nugetizer is mostly a drop-in replacement for "SDK Pack". Seems like a reasonable expectation it should, based on https://docs.microsoft.com/en-us/nuget/reference/msbuild-targets#including-content-in-a-package

[kzu]

Plz provide the csproj with the content laid out by SDK Pack and how nugetizer differs from that. (after just referencing nugetizer, no csproj changes, aka "drop in replacement" :))

Thanks!

[kzu]

Also, perhaps change the issue title since AppendTargetFrameworkToOutputPath doesn't play a role in SDK pack or nugetizer alike? (to the best of my knowledge, at least)

[jzabroski]

It's a good point. I will conduct some tests with SDK Pack and see if it affects the notes mentioned in https://docs.microsoft.com/en-us/nuget/reference/msbuild-targets#including-content-in-a-package

But, for now, can you answer the question as to how I can port my nuspec to Nugetizer? Or is your recommendation I just stick with the Nuspec? I think that's what I am inclined to pick for now. But I figured this would be instructive in helping me understand how things work.

Edit: I did see and take note of your tweet pointing me to https://github.com/devlooped/nugetizer/blob/main/src/NuGetizer.Tasks/NuGetizer.props#L82

[jzabroski]

@kzu When you say AppendTargetFrameworkToOutputPath doesn't play a role in SDK Pack, how do you determine that? See: NuGet/Home#5160 where the author of NuProj, Andrew Arnott, complains about this setting screwing up his packaging. In particular, Rohit says in NuGet/NuGet.Client#1255 that this shipped in Visual Studio v15.3 (and therefore MSBuild v15.3). However, Rohit says 7 months later, in NuGet/Home#6184, that AppendTargetFrameworkToOutputPath screws up dotnet restore and therefore pack should not honor this property. A month later, Rohit added TfmSpecificPackageFile items to overcome this limitation, see here: NuGet/Home#6184 (comment)

THAT SAID... the documentation on Microsoft Docs says that AppendTargetFrameworkToOutputPath is listed under the section "Packaging-related properties" https://docs.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props#appendtargetframeworktooutputpath

It seems like there is some sequential coupling from restore through to publish. I'm honestly confused as to when it should and shouldn't apply, but these are my notes I have compiled so far. My gut feeling is AppendTargetFrameworkToOutputPath should apply, but perhaps there should be sanity checks as well, or else there will be undefined behaviors like the ones Rohit cautions against.

[jzabroski]

@kzu I dug further on this today, and there is an additional edge case (trying to create a minimal repro) where MSBuild /t:pack bundles ItemGroup metadata as packable items for Nuget to process. In that scenario, if you provide a <NuspecFile>package.nuspec</NuspecFile> and the build uses Microsoft.Build.NoTargets, then there is no build output tracked by MSBuild unless you create a nonce file so that Nuget thinks there is something to pack. See: NuGet/Home#3308 (comment)

In addition, AppendTargetFrameworkToOutputPath=false may cause IsTool=true to fail to pack correctly. Specifically, there may not be a DotNetToolSettings.xml file generated when its false. Need to play around with this a bit further to confirm.

In short, Nuget is pretty messed up. There is a lot of sequential coupling form restore target through to publish. Unfortunately, developers need to test the whole process in order to validate the whole command chain works.

[kzu]

But, for now, can you answer the question as to how I can port my nuspec to Nugetizer?

I'm afraid that's beyond the scope of nugetizer (offering a migration guide from .nuspec). Or rather, my limited available time :). Whatever is available in the nuget docs for doing the same to migrate to SDK pack should apply equally, nevertheless. It would depend heavily on how much of a custom nuspec you have.

When you say AppendTargetFrameworkToOutputPath doesn't play a role in SDK Pack, how do you determine that?

Well, I'm saying it basically from the official nuget documentation. That property isn't mentioned anywhere. Publish != packing (entirely or mostly unrelated). If there are bugs in how SDK pack works or doesn't when you use it, I don't know (and I don't particularly care, from nugetizer's point of view). NuGetizer doesn't use it anywhere. Ditto for the other issues you're seeing with DotNetToolSettings.xml and the like (don't even know what that is, LoL).

NOTE: for dotnet-tools packaging, I'm mostly relying on SDK pack. For everything else, I'm using nugetizer. So there may be some parity missing for the former use case. Lower pri for me in particular, since the packing needs are much simpler there for a tool and it's not really worth it to invest too much in nugetizer supporting (specifically) that, unless there are concrete scenarios where SDK pack falls short or breaks entirely (like the many scenarios nugetizer fixes for library authors).

Thanks!

[jzabroski]

In addition, AppendTargetFrameworkToOutputPath=false may cause IsTool=true to fail to pack correctly.

My comment here was wrong; I used IsTool=true rather than PackasTool=true, which is the root cause as to why it packed incorrectly. Bleh. So many variable names to keep track of.

[kzu]

Converted to discussion since it's not clear there's a bug yet that can be worked on. Thanks!

[jzabroski]

I think once we wrap up this discussion we should archive/delete this discussion and make something better in the readme I think. Depends how motivated I am.