Description Class that represents textures in C# code. Use this class to create textures, or to modify existing texture assets. The ImageConversion class provides extension methods to this class that handle image encoding functionality. There are two methods to achieve texture tiling based on size. The first way is adjusting actual texture tiling property whenever a change is detected in scale. The second option is to procedurally generate a plane mesh with many segments all with overlapping UVs. Method #1: Texture Tiling.

Small proof-of-concept idea about streaming raw video from commandline ffmpeg into Unity, using UDP streaming. It works ok for small resolution and low fps rate, but breaks/goes out of sync if any udp data goes missing.

Sources:
https://github.com/unitycoder/ffmpegStreamToUnity

Instructions:
– Download ffmpeg https://ffmpeg.org/download.html
– run from commandline: > ffmpeg -f gdigrab -i desktop -pixel_format rgb8 -video_size 256×256 -vf scale=256:256 -framerate 5 -r 5 -f rawvideo udp://127.0.0.1:8888
– Start the above unity project, and it should display received data

Next steps
– TCP version maybe fixes missed frames/out of sync problems
– Use proper container mpegts or so, to fix issues with udp missed frames etc. but difficult to decode mpeg4 payload data(?)

Unity Texture2d To Rendertexture

$$$ Reward $$$
– First one to post working pull-request (or separate repo) for mpegts/mp4/mpeg4 or handle other proper video stream parsing inside unity (without plugins, or only open source plugins) wins 50usd/paypal, 100usd/paypal : D because i think this could be useful open source tool for the community (to be able to easily stream data from ffmpeg)
– Requirements, parse this data into texture2d:
ffmpeg -f gdigrab -i desktop -r 1 -f mpegts udp://127.0.0.1:8888
// or more options
ffmpeg -f gdigrab -i desktop -pixel_format rgb24 -video_size 512×512 -vf scale=512:512 -framerate 1 -r 1 -f mpegts dp://127.0.0.1:8888
// it doesnt have to be plain UDP, can use rtp, tcp or others too, can use different pixel format too

Related Posts

Unity Texture2d Size

8 Comments + Add Comment

• Hi mgear, I did something similar for OpenGL where I chose to render each pixel with a colored quad. You might be interested in the program here at http://www.alsprogrammingresource.com/video.html

• Hi mgear,

  On Windows, have a look at ViveMediaDecoder by HTC
  https://github.com/ViveSoftware/ViveMediaDecoder

  It is Unity Windows software decoder based on FFmpeg. From the code I see that it should support RTSP. If so you should be able to stream directly from FFmpeg or gstreamer.

  On Linux you could take a look at (my) unity-network-hardware-video-decoder

  It will not get you the job done, it uses custom network protocol but you can get the idea how to do it on Linux and other systems (the rendering part is cross platform).

  By combining the rendering part from UNHVD with decoding from Vive you should be able to get cross platform Unity Player.

  By combining the decoding part from UNHVD with Vive, you should be able to get hardware accelerated Unity Player.

  Kind regards

• Hi, So I’ve tried a lot of things in order to achieve video streaming over UDP…
  bumped into this thread several times:)

  Eventually, salvation came from the attached repo…. it uses AsyncGPUReadbackRequest which is awesome.
  you do need to change the FFmpeg command from inside the code, but it works perfectly and even harness the HW acceleration for decoding.

  anyway enjoy Brother:
  https://github.com/keijiro/FFmpegOut

  and change is this:

  public static FFmpegSession CreateWithOutputPath(
  string outputPath,
  int width, int height, float frameRate,
  FFmpegPreset preset
  )
  {
  return new FFmpegSession(
  “-y -f rawvideo -vcodec rawvideo -pixel_format rgba”
  + ” -colorspace bt709″
  + ” -video_size ” + width + “x” + height
  + ” -framerate ” + frameRate
  + ” -loglevel warning -i – ” + preset.GetOptions()
  //+ ” ”” + outputPath + “””
  //+ ” -f h264 udp://127.0.0.1:6000″
  + ” -f h264 udp://192.168.1.39:6000″
  );
  }

• @mgear, would you have any advice on how to change this from a desktop stream to a stream from a wifi camera over UDP? I’m trying to get the direct feed of a Tello drone camera into Unity.

  I’m just starting with ffmpeg but this is what I’ve tried (and failed with) so far:

  ffmpeg -i udp://[drone’s IP and port here] -r 1 -f mpegts udp://127.0.0.1:8888

  This (probably unsurprisingly) did not work.

Recent posts

Discord Chat

Recent Comments

• on Using RenderDoc with Unity
• on ffmpeg stream raw video into Unity Texture2D
• on Deploy to Android device with wireless connection
• on Vector3 maths for dummies!
• on Waves Shader
• on Matrix Playground Shader
• on Vector3 maths for dummies!
• on Flood Fill Algorithm

@unitycoder_com

Subscribe to Blog via Email

Tag Cloud

Unity Texture To Texture2d

Connect