Відео

"I use ED on Flexowriter via UART on my RISC-V, while standing in full scuba gear in a hammock" Vim? A great little cli editor. "I write Rust/haskell in Vim/emacs on Arch btw" -> a loser.

Hello! I would like to use SAC as shown above in order to beat the Walker2D Environment. I played around with the hyperparameters of the agent based on some research papers for said environment but i cant seem to reach a score above 600-700. I believe that the main reason is the reward scale because I can't understand clearly how it impacts the agent. Also I found out that if I increase the max size that effects negatively the score and the agent cant learn. Any advice on how to proceed. Thank you in advance and thank you for the great content!

Hello Phil, I could not find the repo, please direct me where to find it

Hi, Phil kindly help me with the code for your work. The link there down below does not give me the code (or guide me on how to use the link that you gave).

real nice

I literally loved your videos! That's some amazing work you do. It will help out so many people so immensely!!

Can someone tell me the compatible python version foe this video?

thank for sharing your insight.

Can malware detection project be done using PPO or not.

How would it perform compare with HER (hindsight experience replay)?

I've been using dueling double DQN in my latest attempt to make a self-driving car in GTA V. I'd like to try this framework out at some point.

With your videos I was able to setup vim as you did. Other videos were outdated or didn't quite have it. Thank you!

Great setup. Thanks for sharing. I use vim for everything I can.

HI, did anyone run this in google colab? is there any problem with spawning

I am getting OpenGL related error when trying to visualize by setting evalute flag true.

UT Austin has an online AI masters. Any thoughts on to the program?

nice tutorial thanks Phil, just subscribed udemy to learn your courses there. however here I encountered two problems, might because gym updated configuration recently, bring it up for other's reference: 1, observation = env.reset() should change to observation = env.reset()[0], 2, lunar lander environment is returning 5 values now, added istruncated, so step line should change to observation_, reward, done, istruncated, info = env.step(action)

your videos are good. I'm trying to implement a actor-crtic algorithm for modellimg a process. My process has input flow rate, concentration of species and the output is pH.I'm struggling to implement since i'm a beginner. Kindly make a video tutorial of how to implement actor-critic for a process modelling. It will be helpful for students like us to follow and learn.

Anyone can suggest a way to use this for website optimization (CRO - optimizing fonts and colors, etc)

I dont understand why in your critis_loss (47:14), the "returns" is defined as the advantage + value? Where in the paper is mentioned?

Hello Dr.Phil, I was testing your framework, specifically the dqn and ppo test files, and I found some abnormal behavior. I wanted to know if I could write to you in a specific place about what is happening or if I simply wrote it to you in the github issues

I really loved your point of view very much, it falls within what is called in psychology "positive reinterpretation" as one of the methods of copy with stresses, with your sometimes based on scientific and logical evidence, thank you from the bottom of the heart. Dr. Entesar Abdul Salam Karman

‘The rewards attribution problem...and the result is just poor learning, nobody getting anything done. Typical right?’ So funny XD

Maximum appreciations!

Huge fan here. Will you be considering doing some model based RL such as the DreamerV3 in the future?

Hello, I'm having difficulties running docker-compose, it's my first time on Linux, I decided to change to a real operating system but I don't know if I'm doing things right or not, when I do docker-compose it's telling me that does not find .env, that file is not in the directory.

I think using Neovim instead of Vim is a much better choice. Especially when you go for the plugin ecosystem.

It will be great when they manage to combine reinforcement learning with quantum computing, most likely if they achieve it the jump in learning times will be brutal.

Muchisimas gracias por este contenido de gran valor ❤

Thank you, Doctor. I would like to know the differences between this framework and the stable-baselines library.

The rendering is not working,the window shows empty and its not responding after sometime,the training worked well though😢

Evolution strategies are sometimes used for the same types of problems as reinforcement learning. Is there a reason to prefer RL over ES, other than the popularity bandwagon?

Really great content and a good choice (in my opinion) following the path to Deep RL and QC.

I'll have an opportunity to use this in the coming week. So exciting that the framework is finally ready!

Do you work at FAANG

Another Dr. phil video yay ❤...

Ok I’m 37 and I’m serious considering should I go for a PhD or should I just stop at a masters

me ha sido muy util, muchas gracias

9:35 "The advantage is just a measure of the goodness of each state", that is not correct. The advantage is a measure of how much better a particular action is compared to the average action taken from the same state.

Thanks for update. I wonder how would you go about loading the model, once trained, for "testing"? I've tried for example, loading the model state to q_eval, setting q_eval to ".eval()" mode, using "with torch.no_grad()", then getting the predictions with "model(observation)", but the model/agent doesn't perform as how it was doing during training (it does really bad in comparison).

I have a question... Say, one trains a model, and save its model state for later use... How would one go about loading the model state and performing testing of the agent?.... I've tried coding something (following what I've found on the internet, being, in a nutshell, loading the model state, changing it to eval model, then with torch no grad, selecting the actions greedily), which during training does pretty well at the end of its training (learning was expected), but when I try testing (for instance, to show others its performance), it performs horribly... can anybody help me?

hi sir, could you please tell me the pytorch version you have used in this script

I get the following error when doing "critic_loss.backward()": "RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [256, 1]], which is output 0 of AsStridedBackward0, is at version 2; expected version 1 instead." Has anyone else encountered it?

did someone got this kind of broadcasting error in pytorch, running this code(not exactly copy) UserWarning: Using a target size (torch.Size([100, 100])) that is different to the input size (torch.Size([100])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size. critic_loss = F.mse_loss(q1_old_policy.view(-1),q_target) + F.mse_loss(q2_old_policy.view(-1),q_target),

Hi Phil, I am exploring RL for autonomous driving in my channel by trying to outsource RL to stable baselines3, however I very quickly got into library conflicts in SB3. Now I was looking for ground up builds of RL and found this video. Thanks. I will try to implement it. If you have any other tips, please reach out. Hopefully my videos describe what I would like to do. Thanks.Vadim

Hi @MachineLearningwithPhil, do you plan do some video with multiplos agent as we can see on recent papers like Seek and Hidden from OpenAI?

Thank you for the informative content! Unfortunately, as of April 2024, the code throws the following error: " File "...\main.py", line 31, in <module> action, prob, val = agent.choose_action(observation) File "...\ppo_torch.py", line 136, in choose_action state = T.tensor([observation], dtype=T.float).to(self.actor.device) ValueError: expected sequence of length 4 at dim 2 (got 0)".I know 'CartPole-v0' is outdated but updating it does not solve the problem.

Great videos Phil. One minor improvement you might consider is Loudness normalization to increase the clarity of your voice, as it is kind of low compared to other audio outputs. Keep up the great work!

@MachineLearningwithPhil will this work with a custom tf_py_environment or py_environment?

Try running the 70b llama3 model, see how it goes. On my Mac Studio M2 Ultra it runs faster than ChatGPT. On my PC with a 4090 it's a couple seconds per token. It's night and day (but the PC is faster with smaller models like llama3:8b.) A 4090 doesn't have enough VRAM to run large models.