Sharing common code including Plugins

We have a couple projects that use the same Plugins + libs (some common code that wraps the Plugins and also some general as well, like a math library for example)

We need to have them as a separate from the main repository because we share them between a few projects.

Currently, Plugins and libs are two separate submodules attached to the main project. The problem is that it is not convenient to manage them like that because committing to the Plugins most often than not you also commit to the "libs" to hook up the new .dll for example. One option is to move that "libs" into the Plugins but we would like to avoid this option if possible.

For example, if we just could move the Plugins a layer deeper we could do

 Assets/common/Plugins
              /libs

where "common" is a new submodule that has both Plugins and libs in it (as just files, not sub-submodules). It would solve all our problems.

If you can't actually move plugins (like settings its new path somewhere) then is there some other way to solve this problem?

P.S. subtree is probably also not an option, because as I understand it - it required everyone on the team to avoid committing from different repositories at the same time (like root and Plugins would be in our example), use "git subtree push" instead and also doesn't have a great UI support (which is needed because most of the team are not programmers-like but still use git all the time), so we would preferably live without it to avoid future complications.